From 9e27ba289a2b7bd20aa1b9523566a7e06e76d48b Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 3 Feb 2026 15:15:12 +0100 Subject: [PATCH 01/77] [WIP] ADR EC integration --- .../00012-enterprise-contract-integration.md | 500 ++++++++++++++++++ 1 file changed, 500 insertions(+) create mode 100644 docs/adrs/00012-enterprise-contract-integration.md diff --git a/docs/adrs/00012-enterprise-contract-integration.md b/docs/adrs/00012-enterprise-contract-integration.md new file mode 100644 index 000000000..568ef7b31 --- /dev/null +++ b/docs/adrs/00012-enterprise-contract-integration.md @@ -0,0 +1,500 @@ +# 00012. Enterprise Contract Integration + +Date: 2026-02-03 + +## Status + +PROPOSED + +## Context + +Organizations need to enforce security and compliance policies across their software supply chain to ensure that all software components meet specific criteria for licensing, vulnerabilities, and provenance. Currently, Trustify provides SBOM storage, analysis, and vulnerability tracking, but lacks automated policy enforcement capabilities. + +Manual validation of SBOMs against organizational policies is: + +- Time-consuming and error-prone +- Inconsistent across teams and projects +- Difficult to scale across large numbers of SBOMs +- Lacks audit trails and historical compliance tracking +- Cannot provide real-time feedback on policy violations + +### Requirements + +Users need the ability to: + +1. Automatically validate SBOMs against organizational policies +2. Define and manage multiple policy configurations +3. View compliance status and violation details for each SBOM +4. Track compliance history over time +5. Generate detailed compliance reports for auditing +6. Receive actionable feedback on policy violations + +### Acceptance Criteria + +For this integration to be considered successful, the following criteria must be met: + +1. **Conforma Integration**: Trustify backend integrates with Conforma CLI to execute policy checks against stored SBOMs +2. **Policy Configuration**: Users/administrators can define or reference specific Enterprise Contract policies to enforce via the UI or API +3. **Result Persistence**: Conforma check output (Pass/Fail status and specific violations) is parsed and saved as structured properties on the corresponding SBOM record in the database +4. **Error Handling**: System gracefully handles execution failures (e.g., malformed SBOMs, policy timeouts, Conforma unavailable) and returns appropriate error messages +5. **Visibility**: Users can retrieve the compliance status of an SBOM via the Trustify API or UI, including historical validation results + +### Available Solutions + +**Enterprise Contract (Conforma)** is an open-source policy enforcement tool that: + +- Validates SBOMs against configurable policies +- Supports policies for licensing, vulnerabilities, and provenance +- Provides structured output (JSON) for programmatic consumption +- Integrates with CI/CD pipelines +- Is open source +- Is actively maintained by Red Hat + +**Current state**: Conforma provides a CLI tool but no REST API yet. Future API availability is expected but timeline is undetermined. + +## Decision + +We will integrate Enterprise Contract (Conforma) into Trustify as an optional validation service that: + +1. **Executes Conforma CLI** via async process spawning using `tokio::process::Command` +2. **Stores validation results** as structured data in PostgreSQL with foreign key relationship to SBOMs +3. **Persists detailed reports** in object storage (S3/Minio) for audit trails +4. **Exposes REST API endpoints** for triggering validation and retrieving results +5. **Supports policy configuration** through database-backed policy references + +### System Architecture + +```mermaid +C4Context + title Enterprise Contract Integration - System Context + + Person(user, "Trustify User", "Analyst validating SBOMs") + + System(trustify, "Trustify", "RHTPA") + + System_Ext(conforma, "Conforma", "Enterprise Contract policy validation tool") + System_Ext(s3, "S3", "S3/Minio Storage") + System_Ext(policyRepo, "Policy Repository", "Git repository or storage containing EC policies") + + Rel(user, trustify, "Request Compliance
View compliance status", "API/GUI") + Rel(trustify, conforma, "Executes policy validation", "Spawn Process") + Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") + Rel(trustify, s3, "I3s", S3/Minio Storager, trustify, $offsetX="-30", $offsetY="+20") + + UpdateRelStyle(trustify, conforma, $offsetX="-40") + UpdateRelStyle(user, trustify, $offsetX="-50", $offsetY="20") + + UpdateLayoutConfig($c4ShapeInRow="2", $c4BoundaryInRow="1") +``` + +### Component Diagram - EC Validation Module + +```mermaid +C4Container + title Enterprise Contract Integration - Container Diagram + + Person(user, "Trustify User", "Software engineer or security analyst") + + + Container_Boundary(trustify, "Trustify System") { + Container(webui, "Web UI", "React/TypeScript", "Trustify GUI") + Container(api, "API Gateway", "Actix-web", "REST API endpoints for SBOM
and compliance operations") + ContainerDb(postgres, "PostgreSQL", "DBMS", "Stores SBOM metadata, relationships,
and EC validation results") + Container(ecModule, "EC Validation Module", "Rust", "Orchestrates Conforma CLI
execution and result persistence") + ContainerDb(s3, "Object Storage", "S3/Minio", "Stores SBOM documents and EC reports") + Container(storage, "Storage Service", "Rust", "Manages document storage
(SBOMs, policies results)") + } + + Container_Boundary(conforma, "Conforma System") { + System_Ext(conforma, "Conforma CLI", "External policy validation tool") + System_Ext(policyRepo, "Policy Repository", "Git repository with EC policies") + } + + Rel(user, webui, "Views compliance status", "HTTPS") + Rel(user, api, "Views compliance status", "RESTful") + Rel(webui, api, "API calls", "JSON/HTTPS") + Rel(api, ecModule, "Triggers validation", "Function call") + Rel(ecModule, conforma, "Executes validation", "CLI/Process") + Rel(ecModule, postgres, "Saves validation
results", "SQL") + Rel(ecModule, storage, "Stores EC reports", "Function call") + Rel(storage, s3, "Persists reports", "S3 API") + Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") + + UpdateRelStyle(user, webui, $offsetX="-60", $offsetY="30") + UpdateRelStyle(user, api, $offsetX="-60", $offsetY="-50") + UpdateRelStyle(webui, api, $offsetX="-40", $offsetY="10") + UpdateRelStyle(ecModule, postgres, $offsetX="-40", $offsetY="10") + UpdateRelStyle(storage, s3, $offsetX="-40", $offsetY="10") + + UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="3") +``` + +### Container Diagram + +```mermaid +C4Component + title EC Validation Module - Component Diagram + + Container(api, "API Gateway", "Actix-web", "REST API for EC operations") + + Container_Boundary(ecModule, "EC Validation Module") { + Component(ecEndpoints, "EC Endpoints", "Actix-web handlers", "REST endpoints for
validation operations") + Component(conformaExecutor, "Conforma Executor", "Async Process", "Executes Conforma CLI
and captures output") + Component(policyManager, "Policy Manager", "Configuration", "Manages EC policy
references and configuration") + Component(ecService, "EC Service", "Business logic", "Orchestrates validation workflow") + Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output
into structured data") + Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results to database") + Component(SBOMModel, "SBOM Model", "Data structure", "API models for validation
requests/responses") + } + + Container_Boundary(external, "External Systems") { + System_Ext(conforma, "Conforma CLI", "Enterprise Contract
validation tool") + System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") + } + + Container_Boundary(postgres, "database") { + ContainerDb(postgres, "PostgreSQL", "Database", "Stores validation results") + } + + + Rel(api, ecEndpoints, "POST /sboms/{id}/ec-validate,
GET /sbms/{id}/ec-report", "JSON/HTTPS") + Rel(ecEndpoints, ecService, "validate_sbom()
get_ec_report()", "Function call") + Rel(ecService, policyManager, "get_policy_config()", "Function call") + Rel(ecService, conformaExecutor, "request_validation()", "Function call") + Rel(conformaExecutor, conforma, "Runs CLI command", "Process spawn") + Rel(resultParser, SBOMModel, "Creates models", "Data mapping") + Rel(ecService, resultPersistence, "save_results()", "Function call") + Rel(resultPersistence, postgres, "INSERT validation_results", "SQL") + Rel(ecService, s3, "Store EC report", "S3 API") + + UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") + + UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") +``` + +### Data Model + +Two new tables: + +**`ec_policies`** - Store policy references and metadata (not the policies themselves) + +> **Note**: This table stores references to external policies (Git URLs, OCI registries, etc.) and local configuration metadata. The actual policy definitions remain in their external repositories (GitHub, GitLab, etc.) and are fetched by Conforma at validation time. + +- `id` (UUID, PK) +- `name` (VARCHAR, unique) - User-friendly name for this policy configuration +- `description` (TEXT) - Description of what this policy enforces +- `policy_ref` (VARCHAR) - **External reference**: Git URL, OCI registry, or file path where policy is stored +- `policy_type` (VARCHAR) - 'git', 'oci', 'local' (indicates how Conforma should fetch the policy) +- `configuration` (JSONB) - Additional Conforma parameters (branch, tag, auth credentials, etc.) +- `created_at`, `updated_at` (TIMESTAMP) + +**`ec_validation_results`** - Store validation outcomes + +- `id` (UUID, PK) +- `sbom_id` (UUID, FK → sbom) +- `policy_id` (UUID, FK → ec_policies) +- `status` (VARCHAR) - 'pass', 'fail', 'error' +- `violations` (JSONB) - Structured violation data +- `summary` (JSONB) - Statistics (total checks, passed, failed, warnings) +- `report_url` (VARCHAR) - S3 URL to detailed report +- `executed_at` (TIMESTAMP) +- `execution_duration_ms` (INTEGER) +- `conforma_version` (VARCHAR) +- `error_message` (TEXT) + +### API Endpoints + +``` +POST /api/v2/sboms/{id}/ec-validate # Trigger validation +GET /api/v2/sboms/{id}/ec-report # Get latest validation result +GET /api/v2/sboms/{id}/ec-report/history # Get validation history +GET /api/v2/ec/report/{result_id} # Download detailed report +POST /api/v2/ec/policies # Create policy reference (admin) +GET /api/v2/ec/policies # List policy references +GET /api/v2/ec/policies/{id} # Get policy reference details +PUT /api/v2/ec/policies/{id} # Update policy reference (admin) +DELETE /api/v2/ec/policies/{id} # Delete policy reference (admin) +``` + +### Implementation Approach + +**Phase 1: Backend - CLI Integration** + +- Use async process spawning with proper timeout handling +- Stream stdout/stderr for error capture +- Parse Conforma JSON output into Rust structs +- Handle exit codes (0=pass, 1=fail, 2=error) +- Implement REST API endpoints for validation operations +- Add database schema and migrations +- Create service layer for orchestration + +**Phase 2: Frontend - GUI Development** + +- **SBOM Details View Enhancements**: + - Add "Validate with EC" button/action in SBOM detail page + - Display compliance status badge (Pass/Fail/Error) prominently + - Show latest validation timestamp and policy used +- **Validation Results Display**: + - Create dedicated compliance tab/section in SBOM view + - Display validation summary (total checks, passed, failed, warnings) + - List violations with severity, description, and remediation hints + - Provide expandable/collapsible violation details +- **Validation History**: + - Show historical validation results in timeline view + - Allow filtering by policy, status, and date range + - Display trend charts for compliance over time +- **Policy Management UI** (Admin): + - Policy reference configuration page (store Git URLs, OCI refs, etc.) + - List view of available policy references with descriptions and types + - Policy selection dialog when triggering validation + - Form validation for policy references (validate Git URL format, test connectivity) + - Display policy source (GitHub, GitLab, OCI registry) with clickable links +- **Report Download**: + - Download button for detailed JSON/HTML reports + - Preview detailed report in modal or new page + - Export options (JSON, PDF, HTML) +- **Notifications & Feedback**: + - Loading indicators during validation execution + - Toast notifications for validation completion + - Error messages with actionable guidance + - Progress tracking for long-running validations + +**Phase 3: Future API Migration** + +When Conforma provides a REST API: + +- Implement adapter pattern to switch between CLI and API +- Add feature flag `ec-api-mode` for gradual migration +- Maintain backward compatibility with CLI mode +- Deprecate CLI integration after stable API adoption +- No GUI changes required (transparent backend switch) + +### Module Structure + +``` +modules/ec/ +├── Cargo.toml +└── src/ + ├── lib.rs + ├── endpoints/ + │ └── mod.rs # REST endpoints + ├── model/ + │ ├── mod.rs + │ ├── policy.rs # Policy API models + │ └── validation.rs # Validation result models + ├── service/ + │ ├── mod.rs + │ ├── ec_service.rs # Main orchestration + │ ├── policy_manager.rs # Policy configuration + │ ├── executor.rs # Conforma CLI execution + │ └── result_parser.rs # Output parsing + └── error.rs # Error types +``` + +### Technical Considerations + +**Conforma CLI Execution** + +- Use `tokio::process::Command` for async execution to avoid blocking the runtime +- Stream stdout/stderr for real-time monitoring and logging +- Set execution timeouts (default: 5 minutes, configurable per policy) +- Handle large SBOM files efficiently by streaming to temporary files +- Capture exit codes for error handling (0=pass, 1=fail, 2+=error) +- Sanitize all CLI arguments to prevent injection attacks (use process args array, not shell strings) + +**Policy Management** + +- Support multiple policy types: Git repositories, OCI registries, local file paths +- Cache policy files locally to avoid repeated fetches from external sources +- Validate policy references before execution (check URL format, test connectivity) +- Track policy version/commit when storing validation results for reproducibility +- Support authentication for private policy repositories (tokens, SSH keys) + +**Result Storage** + +- Store structured violations in JSONB for efficient querying (filter by violation type, severity) +- Keep detailed reports in object storage (S3/Minio) to minimize database size +- Implement retention policies for old validation results (configurable, e.g., 90 days) +- Index frequently queried fields (sbom_id, status, executed_at) for performance +- Consider result aggregation for analytics dashboards + +**Error Handling** + +- Distinguish between validation failures (policy violations) and execution errors (CLI crashes) +- Provide actionable error messages to users (e.g., "Policy file not found at URL") +- Implement retry logic for transient failures (network timeouts, temporary service unavailability) +- Log all execution details for debugging (command, arguments, duration, exit code) +- Gracefully degrade when Conforma is unavailable (show cached results, queue for retry) + +**Security** + +- Validate and sanitize all user inputs (policy URLs, SBOM IDs) +- Restrict policy sources to trusted domains (configurable allowlist) +- Store authentication credentials securely (encrypted, never in logs) +- Implement proper authorization checks (only admins can modify policies) +- Audit log all validation executions and policy modifications + +**Performance** + +- Limit concurrent Conforma executions using semaphore (default: 5 concurrent) +- Implement queueing for validation requests during high load +- Cache policy files to reduce external network calls +- Use connection pooling for database operations +- Monitor execution times and resource usage for capacity planning +- Consider horizontal scaling for validation workloads + +## Consequences + +### Positive + +1. **Automated Policy Enforcement**: Organizations can automatically validate SBOMs without manual review +2. **Audit Trail**: Complete history of compliance checks stored in database +3. **Flexibility**: Support multiple policy configurations for different requirements +4. **Integration Ready**: REST API enables integration with CI/CD pipelines +5. **Scalability**: Async execution prevents blocking on long-running validations +6. **Extensibility**: Module design allows future enhancement (webhooks, notifications, etc.) +7. **Open Source**: Conforma is open-source and actively maintained + +### Negative + +1. **External Dependency**: Requires Conforma CLI to be installed on Trustify servers +2. **Process Overhead**: Spawning external processes has performance implications +3. **Error Handling Complexity**: Must handle CLI failures, timeouts, and malformed output +4. **Version Management**: Need to track Conforma version compatibility +5. **Resource Usage**: Multiple concurrent validations may consume significant resources +6. **No Native API**: Until Conforma provides an API the CLI integration is less efficient than native API integration. + +### Risks and Mitigations + +| Risk | Mitigation | +| ----------------------------------- | --------------------------------------------------------------- | +| Conforma CLI unavailable/crashes | Implement health checks, graceful error handling, retry logic | +| Long execution times block requests | Use async execution with configurable timeouts (default: 5 min) | +| Large SBOMs cause memory issues | Stream SBOM to temp file, pass file path to Conforma | +| CLI injection attacks | Sanitize all inputs, use process args array (not shell strings) | +| Version incompatibility | Document required Conforma version, validate on startup | +| Storage costs for reports | Implement retention policies, compress reports | + +### Migration Path + +When Conforma REST API becomes available: + +1. Implement API client alongside CLI executor +2. Add configuration flag to select execution mode +3. Gradually migrate workloads to API mode +4. Deprecate CLI mode after stability period +5. Remove CLI executor in future major version + +### Performance Considerations + +- **Concurrent Limits**: Implement semaphore to limit parallel Conforma executions (default: 5) +- **Timeout**: Default 5-minute timeout, configurable per policy +- **Caching**: Cache policy files to avoid repeated Git fetches +- **Async**: All operations non-blocking using Tokio runtime +- **Streaming**: Stream results incrementally for large reports + +## Alternatives Considered + +### 1. In-Process Policy Engine + +**Pros**: No external dependencies, faster execution, native Rust integration + +**Cons**: Requires reimplementing Enterprise Contract logic, maintenance burden, divergence from upstream + +**Verdict**: Rejected - maintaining policy engine parity with EC would be significant effort + +### 2. Webhook-based Integration + +**Pros**: Decoupled, scalable, easier to manage separate service + +**Cons**: Additional infrastructure, network latency, complexity for simple use case + +**Verdict**: Deferred - could be future enhancement for large-scale deployments + +### 3. Embedded WASM Module + +**Pros**: Sandboxed, portable, no process spawning + +**Cons**: Conforma not available as WASM, would require major upstream changes + +**Verdict**: Rejected - not feasible with current Conforma implementation + +### 4. Batch Processing Queue + +**Pros**: Better resource management, retry logic, priority handling + +**Cons**: Adds complexity, requires queue infrastructure (Redis/RabbitMQ) + +**Verdict**: Deferred - implement if demand increases, start with simple async execution + +## References + +- [Enterprise Contract (Conforma) GitHub](https://github.com/enterprise-contract/ec-cli) +- [Conforma CLI Documentation](https://github.com/enterprise-contract/ec-cli) +- [Design Document](../design/enterprise-contract-integration.md) +- [ADR-00005: Upload API for UI](./00005-ui-upload.md) - Similar async processing pattern +- [ADR-00001: Graph Analytics](./00001-graph-analytics.md) - Database query patterns + +## Implementation Tracking + +### Backend Tasks + +- [ ] Create module structure under `modules/ec` +- [ ] Implement database migrations for new tables (`ec_policies`, `ec_validation_results`) +- [ ] Build Conforma CLI executor with async process handling +- [ ] Create result parser for JSON output +- [ ] Implement policy manager service +- [ ] Implement EC service orchestration layer +- [ ] Add REST API endpoints (validation, results, policies) +- [ ] Write unit and integration tests +- [ ] Add OpenAPI documentation +- [ ] Write deployment documentation (Conforma installation) +- [ ] Add monitoring and metrics (execution times, success rates) +- [ ] Implement error handling and retry logic +- [ ] Add authentication/authorization checks + +### Frontend Tasks + +- [ ] Add "Validate with EC" button to SBOM detail page +- [ ] Create compliance status badge component (Pass/Fail/Error) +- [ ] Implement validation results display with summary statistics +- [ ] Build violations list component with expandable details +- [ ] Create validation history timeline view +- [ ] Add policy selectreference management UI (admin pages) + - [ ] Policy reference list view with search/filter (shows name, external URL, type) + - [ ] Policy reference create/edit form (Git URL, OCI ref, auth config) + - [ ] Policy reference delete confirmation + - [ ] Test policy connectivity button (validate URL is reachable) + - [ ] Policy delete confirmation +- [ ] Add report download functionality (JSON/HTML) +- [ ] Create detailed report preview modal +- [ ] Implement loading indicators for validation execution +- [ ] Add toast notifications for validation completion/errors +- [ ] Create compliance trend charts (optional, if analytics desired) +- [ ] Add help tooltips and documentation links +- [ ] Ensure responsive design for mobile/tablet views +- [ ] Add accessibility features (ARIA labels, keyboard navigation) + +## Open Questions + +1. **Policy Storage & Caching**: Should policies be pulled from external sources (OCI registry, Git repository) at runtime, or should Trustify cache them locally? Should we rely on Conforma's built-in caching or implement our own? How to handle cache invalidation and TTL for cached policies? + +2. **Trigger Mechanism**: Should the EC validation run automatically immediately upon SBOM upload, or will this be a manually triggered/scheduled process? Should users be able to configure auto-validation per SBOM or globally? + +3. **Data Granularity**: Do we store the full raw JSON output from Conforma, or only a summary (Pass/Fail) and a list of specific policy violations to save database space? What's the trade-off between storage cost and audit completeness? + +4. **Conforma Versioning**: Which version of the Conforma CLI are we targeting initially, and how will we handle updates to the policy engine? Should we support multiple Conforma versions concurrently or enforce a single version? + +5. **Policy Source Security**: Should we validate/verify Git repository signatures for policy sources? How to handle private repositories requiring authentication (tokens, SSH keys)? + +6. **Multi-tenancy**: How to isolate policy references per organization in shared Trustify deployments? Should each organization have separate policy namespaces? + +7. **Rate Limiting**: Should we limit validation requests per user/organization to prevent resource exhaustion? What are reasonable limits? + +8. **Notifications**: Should validation results trigger notifications (email, webhook, Slack)? Should notifications be configurable per policy or per SBOM? + +9. **Automatic Re-validation**: Should we re-validate SBOMs when policy definitions are updated in their external repositories? How to detect policy changes (polling, webhooks)? + +10. **UI/UX**: Where in the UI should compliance status be most prominently displayed? Should we show compliance badges on SBOM list views or only in detail views? + +11. **Default Policies**: Should there be system-wide default policy references that apply to all SBOMs unless overridden? How to handle precedence (user-level vs org-level vs system-level)? From 21fe1ef8c64612e5ad2d0c893351d03914146311 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 4 Feb 2026 15:22:17 +0100 Subject: [PATCH 02/77] Fix typos --- .../00012-enterprise-contract-integration.md | 100 +++++++++++++++++- 1 file changed, 95 insertions(+), 5 deletions(-) diff --git a/docs/adrs/00012-enterprise-contract-integration.md b/docs/adrs/00012-enterprise-contract-integration.md index 568ef7b31..7cf7dd544 100644 --- a/docs/adrs/00012-enterprise-contract-integration.md +++ b/docs/adrs/00012-enterprise-contract-integration.md @@ -79,7 +79,7 @@ C4Context Rel(user, trustify, "Request Compliance
View compliance status", "API/GUI") Rel(trustify, conforma, "Executes policy validation", "Spawn Process") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") - Rel(trustify, s3, "I3s", S3/Minio Storager, trustify, $offsetX="-30", $offsetY="+20") + Rel(trustify, s3, "3s", S3/Minio Storager, trustify, $offsetX="-30", $offsetY="+20") UpdateRelStyle(trustify, conforma, $offsetX="-40") UpdateRelStyle(user, trustify, $offsetX="-50", $offsetY="20") @@ -102,7 +102,7 @@ C4Container ContainerDb(postgres, "PostgreSQL", "DBMS", "Stores SBOM metadata, relationships,
and EC validation results") Container(ecModule, "EC Validation Module", "Rust", "Orchestrates Conforma CLI
execution and result persistence") ContainerDb(s3, "Object Storage", "S3/Minio", "Stores SBOM documents and EC reports") - Container(storage, "Storage Service", "Rust", "Manages document storage
(SBOMs, policies results)") + Container(storage, "Storage Service", "Rust", "Manages document storage
(SBOMs, policy results)") } Container_Boundary(conforma, "Conforma System") { @@ -157,7 +157,7 @@ C4Component } - Rel(api, ecEndpoints, "POST /sboms/{id}/ec-validate,
GET /sbms/{id}/ec-report", "JSON/HTTPS") + Rel(api, ecEndpoints, "POST /sboms/{id}/ec-validate,
GET /sboms/{id}/ec-report", "JSON/HTTPS") Rel(ecEndpoints, ecService, "validate_sbom()
get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") Rel(ecService, conformaExecutor, "request_validation()", "Function call") @@ -172,6 +172,96 @@ C4Component UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` +```mermaid +sequenceDiagram + autonumber + actor User + participant UI as Trustify UI + participant API as Trustify API + participant VS as Validation Service + participant PM as Policy Manager + participant DB as PostgreSQL + participant S3 as Object Storage + participant Conf as Conforma CLI + + User->>UI: Request SBOM validation for policy + UI->>API: POST /api/v2/sbom/{sbom_id}/validate + Note over UI,API: Request body: {policy_id} + + API->>VS: validate_sbom_against_policy(sbom_id, policy_id) + + rect rgb(42, 48, 53) + Note over VS,PM: Policy Resolution Phase + VS->>PM: get_policy_configuration(policy_id) + PM->>DB: SELECT * FROM ec_policies WHERE id = ? + DB-->>PM: Policy configuration + alt Policy not found + PM-->>VS: Error: PolicyNotFound + VS-->>API: 404 Not Found + API-->>UI: Policy not found error + UI-->>User: Display error: "Policy does not exist" + end + PM-->>VS: PolicyConfig {name, policy_ref, version} + end + + rect rgb(68, 66, 62) + Note over VS,S3: SBOM Retrieval Phase + VS->>DB: SELECT * FROM sbom WHERE id = ? + DB-->>VS: SBOM metadata + alt SBOM not found + VS-->>API: 404 Not Found + API-->>UI: SBOM not found error + UI-->>User: Display error: "SBOM does not exist" + end + + VS->>S3: retrieve_sbom_document(sbom_id) + S3-->>VS: SBOM document (JSON/XML) + end + + rect rgb(42, 48, 53) + Note over VS,Conf: Validation Execution Phase + VS->>VS: Create temp files for SBOM and policy + VS->>Conf: spawn: conforma validate
--policy={policy_ref}
--sbom={sbom_file}
--output=json + + alt Validation passes + Conf-->>VS: Exit code: 0
JSON: {result: "PASS", violations: []} + VS->>VS: Parse validation results + VS->>DB: INSERT INTO ec_validation_results
(sbom_id, policy_id, status='passed',
violations=[], timestamp) + DB-->>VS: result_id + VS->>S3: store_validation_report(result_id, full_json) + S3-->>VS: report_url + VS->>DB: UPDATE ec_validation_results
SET report_url = ? + DB-->>VS: Updated + VS-->>API: ValidationResult {status: "passed",
violations: [], report_url} + API-->>UI: 200 OK {passed: true, violations: 0} + UI-->>User: ✓ SBOM passes policy validation + + else Validation fails with violations + Conf-->>VS: Exit code: 1
JSON: {result: "FAIL",
violations: [{rule, severity, message}]} + VS->>VS: Parse validation results + VS->>DB: INSERT INTO ec_validation_results
(sbom_id, policy_id, status='failed',
violations=json, timestamp) + DB-->>VS: result_id + VS->>S3: store_validation_report(result_id, full_json) + S3-->>VS: report_url + VS->>DB: UPDATE ec_validation_results
SET report_url = ? + DB-->>VS: Updated + VS-->>API: ValidationResult {status: "failed",
violations: [...], report_url} + API-->>UI: 200 OK {passed: false, violations: [...]} + UI-->>User: ✗ SBOM violates policy
Show violation details + + else Conforma execution error + Conf-->>VS: Exit code: 2
stderr: "Policy file not found" + VS->>DB: INSERT INTO ec_validation_results
(sbom_id, policy_id, status='error',
error_message=stderr) + DB-->>VS: result_id + VS-->>API: Error: ValidationExecutionFailed + API-->>UI: 500 Internal Server Error + UI-->>User: Display error: "Validation failed to execute" + end + end + + VS->>VS: Cleanup temp files +``` + ### Data Model Two new tables: @@ -409,7 +499,7 @@ When Conforma REST API becomes available: **Cons**: Additional infrastructure, network latency, complexity for simple use case -**Verdict**: Deferred - could be future enhancement for large-scale deployments +**Verdict**: Deferred - could be future enhancements for large-scale deployments ### 3. Embedded WASM Module @@ -460,7 +550,7 @@ When Conforma REST API becomes available: - [ ] Implement validation results display with summary statistics - [ ] Build violations list component with expandable details - [ ] Create validation history timeline view -- [ ] Add policy selectreference management UI (admin pages) +- [ ] Add policy reference management UI (admin pages) - [ ] Policy reference list view with search/filter (shows name, external URL, type) - [ ] Policy reference create/edit form (Git URL, OCI ref, auth config) - [ ] Policy reference delete confirmation From 73f53206fec81c13db3c8edfa3764fa2bb4b20eb Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 9 Feb 2026 15:39:33 +0100 Subject: [PATCH 03/77] Fix broken/missing/redundant links;more details --- ...-integration.md => 00014-enterprise-contract-integration.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename docs/adrs/{00012-enterprise-contract-integration.md => 00014-enterprise-contract-integration.md} (99%) diff --git a/docs/adrs/00012-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md similarity index 99% rename from docs/adrs/00012-enterprise-contract-integration.md rename to docs/adrs/00014-enterprise-contract-integration.md index 7cf7dd544..5ca3ea1a2 100644 --- a/docs/adrs/00012-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -1,4 +1,4 @@ -# 00012. Enterprise Contract Integration +# 00014. Enterprise Contract Integration Date: 2026-02-03 From e674f5fb43ca40f4ee5853e73195beae0c15ca16 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 9 Feb 2026 16:09:54 +0100 Subject: [PATCH 04/77] User realistic parameters for conforma CLI --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 5ca3ea1a2..64f463576 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -221,7 +221,7 @@ sequenceDiagram rect rgb(42, 48, 53) Note over VS,Conf: Validation Execution Phase VS->>VS: Create temp files for SBOM and policy - VS->>Conf: spawn: conforma validate
--policy={policy_ref}
--sbom={sbom_file}
--output=json + VS->>Conf: spawn: conforma validate
input --file "$1" --policy
--output json --show-successes
--info alt Validation passes Conf-->>VS: Exit code: 0
JSON: {result: "PASS", violations: []} From 5fa78732205bff8e13d81fbb52364721358e53ac Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 9 Feb 2026 16:12:29 +0100 Subject: [PATCH 05/77] Offset link text --- docs/adrs/00014-enterprise-contract-integration.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 64f463576..cfd361368 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -168,6 +168,7 @@ C4Component Rel(ecService, s3, "Store EC report", "S3 API") UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") + UpdateRelStyle(ecService, policyManager, $offsetX="-50", $offsetY="-50") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` From 5003d084d66df1f0e459cb68dd36dcb466a59c29 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 16 Feb 2026 10:10:31 +0100 Subject: [PATCH 06/77] Consolidate trade-offs --- .../00014-enterprise-contract-integration.md | 51 +++++-------------- 1 file changed, 13 insertions(+), 38 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index cfd361368..dfd4ca703 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -79,7 +79,7 @@ C4Context Rel(user, trustify, "Request Compliance
View compliance status", "API/GUI") Rel(trustify, conforma, "Executes policy validation", "Spawn Process") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") - Rel(trustify, s3, "3s", S3/Minio Storager, trustify, $offsetX="-30", $offsetY="+20") + Rel(trustify, s3, "Stores reports", "S3 API") UpdateRelStyle(trustify, conforma, $offsetX="-40") UpdateRelStyle(user, trustify, $offsetX="-50", $offsetY="20") @@ -446,43 +446,19 @@ modules/ec/ 6. **Extensibility**: Module design allows future enhancement (webhooks, notifications, etc.) 7. **Open Source**: Conforma is open-source and actively maintained -### Negative +### Trade-offs and Risks -1. **External Dependency**: Requires Conforma CLI to be installed on Trustify servers -2. **Process Overhead**: Spawning external processes has performance implications -3. **Error Handling Complexity**: Must handle CLI failures, timeouts, and malformed output -4. **Version Management**: Need to track Conforma version compatibility -5. **Resource Usage**: Multiple concurrent validations may consume significant resources -6. **No Native API**: Until Conforma provides an API the CLI integration is less efficient than native API integration. - -### Risks and Mitigations - -| Risk | Mitigation | -| ----------------------------------- | --------------------------------------------------------------- | -| Conforma CLI unavailable/crashes | Implement health checks, graceful error handling, retry logic | -| Long execution times block requests | Use async execution with configurable timeouts (default: 5 min) | -| Large SBOMs cause memory issues | Stream SBOM to temp file, pass file path to Conforma | -| CLI injection attacks | Sanitize all inputs, use process args array (not shell strings) | -| Version incompatibility | Document required Conforma version, validate on startup | -| Storage costs for reports | Implement retention policies, compress reports | - -### Migration Path - -When Conforma REST API becomes available: - -1. Implement API client alongside CLI executor -2. Add configuration flag to select execution mode -3. Gradually migrate workloads to API mode -4. Deprecate CLI mode after stability period -5. Remove CLI executor in future major version - -### Performance Considerations - -- **Concurrent Limits**: Implement semaphore to limit parallel Conforma executions (default: 5) -- **Timeout**: Default 5-minute timeout, configurable per policy -- **Caching**: Cache policy files to avoid repeated Git fetches -- **Async**: All operations non-blocking using Tokio runtime -- **Streaming**: Stream results incrementally for large reports +| Trade-off / Risk | Impact | Mitigation | +| ------------------------------- | ---------------------------------------- | -------------------------------------------------------------------------- | +| External CLI dependency | Requires Conforma installed on servers | Health checks, graceful error handling, retry logic | +| Process spawning overhead | Performance implications per validation | Async execution with configurable timeouts (default: 5 min) | +| Error handling complexity | CLI failures, timeouts, malformed output | Distinguish validation failures from execution errors; actionable messages | +| Version management | Conforma version compatibility | Document required version, validate on startup | +| Resource usage under load | Concurrent validations consume resources | Semaphore limits (default: 5), queueing, monitoring | +| No native API yet | CLI less efficient than REST integration | Adapter pattern for future API migration (see Phase 3) | +| Large SBOMs cause memory issues | Out-of-memory during validation | Stream SBOM to temp file, pass file path to Conforma | +| CLI injection attacks | Security vulnerability | Sanitize all inputs, use process args array (not shell strings) | +| Storage costs for reports | Growing storage over time | Retention policies, report compression | ## Alternatives Considered @@ -556,7 +532,6 @@ When Conforma REST API becomes available: - [ ] Policy reference create/edit form (Git URL, OCI ref, auth config) - [ ] Policy reference delete confirmation - [ ] Test policy connectivity button (validate URL is reachable) - - [ ] Policy delete confirmation - [ ] Add report download functionality (JSON/HTML) - [ ] Create detailed report preview modal - [ ] Implement loading indicators for validation execution From ef0d79d03433c361407c72e48e87f126455d1881 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Feb 2026 10:20:48 +0100 Subject: [PATCH 07/77] Consolidate, remove redundancy, more focus on details --- .../00014-enterprise-contract-integration.md | 549 ++++++------------ 1 file changed, 175 insertions(+), 374 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index dfd4ca703..5036577b5 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -8,15 +8,9 @@ PROPOSED ## Context -Organizations need to enforce security and compliance policies across their software supply chain to ensure that all software components meet specific criteria for licensing, vulnerabilities, and provenance. Currently, Trustify provides SBOM storage, analysis, and vulnerability tracking, but lacks automated policy enforcement capabilities. +Trustify provides SBOM storage, analysis, and vulnerability tracking but lacks automated policy enforcement. Organizations need to validate SBOMs against security and compliance policies (licensing, vulnerabilities, provenance) without relying on manual, inconsistent review processes. -Manual validation of SBOMs against organizational policies is: - -- Time-consuming and error-prone -- Inconsistent across teams and projects -- Difficult to scale across large numbers of SBOMs -- Lacks audit trails and historical compliance tracking -- Cannot provide real-time feedback on policy violations +Enterprise Contract (Conforma) is an open-source policy enforcement tool actively maintained by Red Hat. It validates SBOMs against configurable policies and produces structured JSON output. Currently it provides only a CLI; a REST API is planned but with no committed timeline. ### Requirements @@ -29,38 +23,81 @@ Users need the ability to: 5. Generate detailed compliance reports for auditing 6. Receive actionable feedback on policy violations -### Acceptance Criteria +## Decision + +We will integrate Conforma into Trustify as an optional validation service by spawning the Conforma CLI asynchronously. +Validation is manually triggered — not automatic on SBOM upload. +Validation on upload is deferred to a follow-up version. -For this integration to be considered successful, the following criteria must be met: +What is stored where -1. **Conforma Integration**: Trustify backend integrates with Conforma CLI to execute policy checks against stored SBOMs -2. **Policy Configuration**: Users/administrators can define or reference specific Enterprise Contract policies to enforce via the UI or API -3. **Result Persistence**: Conforma check output (Pass/Fail status and specific violations) is parsed and saved as structured properties on the corresponding SBOM record in the database -4. **Error Handling**: System gracefully handles execution failures (e.g., malformed SBOMs, policy timeouts, Conforma unavailable) and returns appropriate error messages -5. **Visibility**: Users can retrieve the compliance status of an SBOM via the Trustify API or UI, including historical validation results +- PostgreSQL: validation status, structured violations (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, status, executed_at. +- S3/Minio: full raw Conforma JSON report, linked from the DB row via report_url. Keeps DB rows small while preserving audit completeness. +- Not stored: the policy definitions themselves. ec_policies stores references (URLs, OCI refs) that Conforma fetches at runtime. -### Available Solutions +Storing full JSON in S3 rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB violations JSONB holds enough structure for filtering and dashboards without duplicating the full payload. -**Enterprise Contract (Conforma)** is an open-source policy enforcement tool that: +### Deferred Validation -- Validates SBOMs against configurable policies -- Supports policies for licensing, vulnerabilities, and provenance -- Provides structured output (JSON) for programmatic consumption -- Integrates with CI/CD pipelines -- Is open source -- Is actively maintained by Red Hat +I has been decided that uploaded SBOMs start in "Pending" status and are not discoverable until validated. +EC validation is one mechanism by which an SBOM can move from "Pending" to "Accepted" or "Rejected". -**Current state**: Conforma provides a CLI tool but no REST API yet. Future API availability is expected but timeline is undetermined. +Concretely: -## Decision + An SBOM in "Pending" can be submitted for EC validation. + A passing EC result transitions the SBOM to "Accepted". + A failing EC result transitions it to "Rejected", with the violation details linked. + An EC execution error (CLI crash, policy fetch failure) does not change the SBOM's policy status — the SBOM stays Pending and the error is surfaced separately, so it doesn't silently block an SBOM. + +## Consequences + +Integrating via CLI spawning rather than a native API introduces an external process dependency that adds operational overhead (Conforma must be installed and version-pinned on every server) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma API exists yet. The executor is built behind an adapter interface so the implementation can be swapped for a REST client in Phase 3 without changes to the service layer or API. + +### Trade-off/Risk & Mitigation + +- Conforma must be installed on servers + - Health check on startup + - graceful degradation showing cached results +- Process spawn overhead per validation + - Async execution; configurable timeout (default 5 min) +- CLI injection attacks + - Args array only, never shell strings; all user inputs sanitized +- Version compatibility + - Document required Conforma version; validate on startup +- Concurrent load exhausting resources + - Semaphore (default: 5) + - 429 on exhaustion +- No native API yet + - Adapter pattern for future migration (Phase 3) +- Large SBOMs causing OOM + - Stream to temp file + - pass path to Conforma +- Growing S3 storage costs + - Retention policy (90-day default) + +### Alternatives Considered + +#### In-Process Policy Engine: Rejected + +Reimplementing Enterprise Contract logic in Rust would diverge from upstream and create significant maintenance burden. + +#### Webhook-based Integration: Deferred -We will integrate Enterprise Contract (Conforma) into Trustify as an optional validation service that: +Decouples validation into a separate service, which is better for large-scale deployments but adds infrastructure complexity premature for initial scope. -1. **Executes Conforma CLI** via async process spawning using `tokio::process::Command` -2. **Stores validation results** as structured data in PostgreSQL with foreign key relationship to SBOMs -3. **Persists detailed reports** in object storage (S3/Minio) for audit trails -4. **Exposes REST API endpoints** for triggering validation and retrieving results -5. **Supports policy configuration** through database-backed policy references +#### Embedded WASM Module: Rejected + +Conforma is not available as WASM and would require major upstream changes. + +#### Batch Processing Queue: Deferred + +A Redis/RabbitMQ queue would improve retry handling and priority management; implement if the 429-based rejection approach proves insufficient under real load. + +### Future API Migration + +When Conforma provides a REST API, the executor.rs adapter is replaced with an HTTP client. A feature flag (ec-api-mode) allows gradual migration. No changes to service layer, API endpoints, or UI are required. + +## The solution ### System Architecture @@ -138,227 +175,167 @@ C4Component Container(api, "API Gateway", "Actix-web", "REST API for EC operations") Container_Boundary(ecModule, "EC Validation Module") { - Component(ecEndpoints, "EC Endpoints", "Actix-web handlers", "REST endpoints for
validation operations") - Component(conformaExecutor, "Conforma Executor", "Async Process", "Executes Conforma CLI
and captures output") - Component(policyManager, "Policy Manager", "Configuration", "Manages EC policy
references and configuration") + Component(ecEndpoints, "EC Endpoints", "Actix-web handlers", "REST endpoints for validation operations") + Component(conformaExecutor, "Conforma Executor", "Async Process", "Spawns Conforma CLI and captures output") Component(ecService, "EC Service", "Business logic", "Orchestrates validation workflow") - Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output
into structured data") - Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results to database") - Component(SBOMModel, "SBOM Model", "Data structure", "API models for validation
requests/responses") + Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output into structured data") + Component(policyManager, "Policy Manager", "Configuration", "Manages EC policy references") + Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results") } - Container_Boundary(external, "External Systems") { - System_Ext(conforma, "Conforma CLI", "Enterprise Contract
validation tool") - System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") + Container_Boundary(external, "External System") { + System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") } - Container_Boundary(postgres, "database") { - ContainerDb(postgres, "PostgreSQL", "Database", "Stores validation results") + Container_Boundary(dbms, "Database") { + ContainerDb(postgres, "PostgreSQL", "Database", "Stores validation results and policy references") + } + Container_Boundary(storage, "S3 System") { + System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") } - - Rel(api, ecEndpoints, "POST /sboms/{id}/ec-validate,
GET /sboms/{id}/ec-report", "JSON/HTTPS") - Rel(ecEndpoints, ecService, "validate_sbom()
get_ec_report()", "Function call") + Rel(api, ecEndpoints, "POST /sboms/{id}/ec-validate,\nGET /sboms/{id}/ec-report", "JSON/HTTPS") + Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") + Rel(policyManager, postgres, "SELECT ec_policies", "SQL") Rel(ecService, conformaExecutor, "request_validation()", "Function call") Rel(conformaExecutor, conforma, "Runs CLI command", "Process spawn") - Rel(resultParser, SBOMModel, "Creates models", "Data mapping") + Rel(ecService, resultParser, "parse_output()", "Function call") Rel(ecService, resultPersistence, "save_results()", "Function call") - Rel(resultPersistence, postgres, "INSERT validation_results", "SQL") + Rel(resultPersistence, postgres, "INSERT ec_validation_results", "SQL") Rel(ecService, s3, "Store EC report", "S3 API") UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") - UpdateRelStyle(ecService, policyManager, $offsetX="-50", $offsetY="-50") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") + ``` +### The main sequence Diagram + ```mermaid sequenceDiagram autonumber actor User - participant UI as Trustify UI participant API as Trustify API - participant VS as Validation Service + participant VS as EC Service participant PM as Policy Manager participant DB as PostgreSQL participant S3 as Object Storage participant Conf as Conforma CLI - User->>UI: Request SBOM validation for policy - UI->>API: POST /api/v2/sbom/{sbom_id}/validate - Note over UI,API: Request body: {policy_id} - - API->>VS: validate_sbom_against_policy(sbom_id, policy_id) - - rect rgb(42, 48, 53) - Note over VS,PM: Policy Resolution Phase - VS->>PM: get_policy_configuration(policy_id) - PM->>DB: SELECT * FROM ec_policies WHERE id = ? - DB-->>PM: Policy configuration - alt Policy not found - PM-->>VS: Error: PolicyNotFound - VS-->>API: 404 Not Found - API-->>UI: Policy not found error - UI-->>User: Display error: "Policy does not exist" - end - PM-->>VS: PolicyConfig {name, policy_ref, version} - end + User->>API: POST /api/v2/sbom/{sbom_id}/ec-validate {policy_id} - rect rgb(68, 66, 62) - Note over VS,S3: SBOM Retrieval Phase - VS->>DB: SELECT * FROM sbom WHERE id = ? - DB-->>VS: SBOM metadata - alt SBOM not found - VS-->>API: 404 Not Found - API-->>UI: SBOM not found error - UI-->>User: Display error: "SBOM does not exist" - end - - VS->>S3: retrieve_sbom_document(sbom_id) - S3-->>VS: SBOM document (JSON/XML) + VS->>PM: get_policy_configuration(policy_id) + PM->>DB: SELECT * FROM ec_policies WHERE id = ? + alt Policy not found + PM-->>VS: Error: PolicyNotFound + VS-->>API: 404 Not Found end + PM-->>VS: PolicyConfig {name, policy_ref, type} - rect rgb(42, 48, 53) - Note over VS,Conf: Validation Execution Phase - VS->>VS: Create temp files for SBOM and policy - VS->>Conf: spawn: conforma validate
input --file "$1" --policy
--output json --show-successes
--info - - alt Validation passes - Conf-->>VS: Exit code: 0
JSON: {result: "PASS", violations: []} - VS->>VS: Parse validation results - VS->>DB: INSERT INTO ec_validation_results
(sbom_id, policy_id, status='passed',
violations=[], timestamp) - DB-->>VS: result_id - VS->>S3: store_validation_report(result_id, full_json) - S3-->>VS: report_url - VS->>DB: UPDATE ec_validation_results
SET report_url = ? - DB-->>VS: Updated - VS-->>API: ValidationResult {status: "passed",
violations: [], report_url} - API-->>UI: 200 OK {passed: true, violations: 0} - UI-->>User: ✓ SBOM passes policy validation - - else Validation fails with violations - Conf-->>VS: Exit code: 1
JSON: {result: "FAIL",
violations: [{rule, severity, message}]} - VS->>VS: Parse validation results - VS->>DB: INSERT INTO ec_validation_results
(sbom_id, policy_id, status='failed',
violations=json, timestamp) - DB-->>VS: result_id - VS->>S3: store_validation_report(result_id, full_json) - S3-->>VS: report_url - VS->>DB: UPDATE ec_validation_results
SET report_url = ? - DB-->>VS: Updated - VS-->>API: ValidationResult {status: "failed",
violations: [...], report_url} - API-->>UI: 200 OK {passed: false, violations: [...]} - UI-->>User: ✗ SBOM violates policy
Show violation details - - else Conforma execution error - Conf-->>VS: Exit code: 2
stderr: "Policy file not found" - VS->>DB: INSERT INTO ec_validation_results
(sbom_id, policy_id, status='error',
error_message=stderr) - DB-->>VS: result_id - VS-->>API: Error: ValidationExecutionFailed - API-->>UI: 500 Internal Server Error - UI-->>User: Display error: "Validation failed to execute" - end + VS->>DB: SELECT * FROM sbom WHERE id = ? + alt SBOM not found + VS-->>API: 404 Not Found + end + VS->>S3: retrieve_sbom_document(sbom_id) + S3-->>VS: SBOM document (JSON/XML) + + VS->>VS: Write SBOM to temp file + VS->>Conf: conforma validate input --file "$SBOM_PATH" --policy --output json --show-successes --info + + alt Exit code 0 — pass + Conf-->>VS: {result: "PASS", violations: []} + VS->>DB: INSERT ec_validation_results (status='pass', violations=[], ...) + VS->>S3: store_validation_report(result_id, full_json) + VS->>DB: UPDATE SET report_url = ? + VS->>DB: UPDATE sbom SET policy_status = 'Accepted' + VS-->>API: 200 {passed: true, violations: 0, report_url} + else Exit code 1 — fail (policy violations) + Conf-->>VS: {result: "FAIL", violations: [{rule, severity, message}]} + VS->>DB: INSERT ec_validation_results (status='fail', violations=json, ...) + VS->>S3: store_validation_report(result_id, full_json) + VS->>DB: UPDATE SET report_url = ? + VS->>DB: UPDATE sbom SET policy_status = 'Rejected' + VS-->>API: 200 {passed: false, violations: [...], report_url} + else Exit code 2+ — execution error + Conf-->>VS: stderr: "Policy file not found" + VS->>DB: INSERT ec_validation_results (status='error', error_message=stderr) + Note over VS,DB: SBOM policy_status unchanged — stays Pending + VS-->>API: 500 {error: "Validation failed to execute", detail: stderr} end VS->>VS: Cleanup temp files ``` -### Data Model - -Two new tables: +### The Data Model -**`ec_policies`** - Store policy references and metadata (not the policies themselves) - -> **Note**: This table stores references to external policies (Git URLs, OCI registries, etc.) and local configuration metadata. The actual policy definitions remain in their external repositories (GitHub, GitLab, etc.) and are fetched by Conforma at validation time. +**`ec_policies`** - Stores references to external policies, not the policies themselves - `id` (UUID, PK) -- `name` (VARCHAR, unique) - User-friendly name for this policy configuration -- `description` (TEXT) - Description of what this policy enforces -- `policy_ref` (VARCHAR) - **External reference**: Git URL, OCI registry, or file path where policy is stored -- `policy_type` (VARCHAR) - 'git', 'oci', 'local' (indicates how Conforma should fetch the policy) -- `configuration` (JSONB) - Additional Conforma parameters (branch, tag, auth credentials, etc.) +- `name` (VARCHAR, unique) - User-friendly name label +- `description` (TEXT) - What this policy enforces +- `policy_ref` (VARCHAR) - Git URL, OCI registry, or file path +- `policy_type` (VARCHAR) - 'git', 'oci', 'local' +- `configuration` (JSONB) - Branch, tag, auth credentials, etc. - `created_at`, `updated_at` (TIMESTAMP) -**`ec_validation_results`** - Store validation outcomes +**`ec_validation_results`** - one row per validation execution - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → ec_policies) - `status` (VARCHAR) - 'pass', 'fail', 'error' -- `violations` (JSONB) - Structured violation data -- `summary` (JSONB) - Statistics (total checks, passed, failed, warnings) +- `violations` (JSONB) - Structured violation data for querying +- `summary` (JSONB) - Total checks, passed, failed, warnings - `report_url` (VARCHAR) - S3 URL to detailed report - `executed_at` (TIMESTAMP) - `execution_duration_ms` (INTEGER) -- `conforma_version` (VARCHAR) -- `error_message` (TEXT) +- `conforma_version` (VARCHAR) - For reproducibility +- `error_message` (TEXT) - Populated only on error status ### API Endpoints ``` -POST /api/v2/sboms/{id}/ec-validate # Trigger validation -GET /api/v2/sboms/{id}/ec-report # Get latest validation result -GET /api/v2/sboms/{id}/ec-report/history # Get validation history -GET /api/v2/ec/report/{result_id} # Download detailed report -POST /api/v2/ec/policies # Create policy reference (admin) -GET /api/v2/ec/policies # List policy references -GET /api/v2/ec/policies/{id} # Get policy reference details -PUT /api/v2/ec/policies/{id} # Update policy reference (admin) -DELETE /api/v2/ec/policies/{id} # Delete policy reference (admin) +POST /api/v2/sboms/{id}/ec-validate # Trigger validation +GET /api/v2/sboms/{id}/ec-report # Get latest validation result +GET /api/v2/sboms/{id}/ec-report/history # Get validation history +GET /api/v2/ec/report/{result_id} # Download detailed report from S3 + +POST /api/v2/ec/policies # Create policy reference (admin) +GET /api/v2/ec/policies # List policy references +GET /api/v2/ec/policies/{id} # Get policy reference +PUT /api/v2/ec/policies/{id} # Update policy reference (admin) +DELETE /api/v2/ec/policies/{id} # Delete policy reference (admin) ``` -### Implementation Approach - -**Phase 1: Backend - CLI Integration** - -- Use async process spawning with proper timeout handling -- Stream stdout/stderr for error capture -- Parse Conforma JSON output into Rust structs -- Handle exit codes (0=pass, 1=fail, 2=error) -- Implement REST API endpoints for validation operations -- Add database schema and migrations -- Create service layer for orchestration - -**Phase 2: Frontend - GUI Development** - -- **SBOM Details View Enhancements**: - - Add "Validate with EC" button/action in SBOM detail page - - Display compliance status badge (Pass/Fail/Error) prominently - - Show latest validation timestamp and policy used -- **Validation Results Display**: - - Create dedicated compliance tab/section in SBOM view - - Display validation summary (total checks, passed, failed, warnings) - - List violations with severity, description, and remediation hints - - Provide expandable/collapsible violation details -- **Validation History**: - - Show historical validation results in timeline view - - Allow filtering by policy, status, and date range - - Display trend charts for compliance over time -- **Policy Management UI** (Admin): - - Policy reference configuration page (store Git URLs, OCI refs, etc.) - - List view of available policy references with descriptions and types - - Policy selection dialog when triggering validation - - Form validation for policy references (validate Git URL format, test connectivity) - - Display policy source (GitHub, GitLab, OCI registry) with clickable links -- **Report Download**: - - Download button for detailed JSON/HTML reports - - Preview detailed report in modal or new page - - Export options (JSON, PDF, HTML) -- **Notifications & Feedback**: - - Loading indicators during validation execution - - Toast notifications for validation completion - - Error messages with actionable guidance - - Progress tracking for long-running validations - -**Phase 3: Future API Migration** - -When Conforma provides a REST API: - -- Implement adapter pattern to switch between CLI and API -- Add feature flag `ec-api-mode` for gradual migration -- Maintain backward compatibility with CLI mode -- Deprecate CLI integration after stable API adoption -- No GUI changes required (transparent backend switch) +### Technical Considerations + +#### Conforma CLI Execution + +Conforma is invoked via tokio::process::Command to avoid blocking the async runtime. All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are streamed to a temp file and passed by path rather than piped via stdin, which avoids OOM issues with the parent process. + +Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected failure, not an error), 2+ = execution error. It is important to distinguish 1 from 2+ in error handling — a policy violation is a valid result that should be surfaced to the user, not treated as a system failure. + +Temp files (SBOM, any cached policy material) are cleaned up in a finally-equivalent block regardless of execution outcome, including on timeout. + +#### Concurrency and Backpressure + +Concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, incoming validation requests return 429 Too Many Requests immediately rather than queuing or blocking indefinitely. This makes the capacity limit explicit to callers (e.g. CI pipelines can implement their own retry with backoff). If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. + +#### Policy Management + +ec_policies stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and must be encrypted at rest; they are never logged. + +Policy version/commit hash is recorded in the conforma_version field of each result row, enabling reproducibility and audit. + +#### Multi-tenancy + +Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. + +#### Data Retention + +Validation results accumulate over time. A retention policy (default: 90 days, configurable) should be implemented to prune old ec_validation_results rows and their corresponding S3 reports. This is deferred to the implementation phase but must be included before production rollout. ### Module Structure @@ -382,185 +359,9 @@ modules/ec/ └── error.rs # Error types ``` -### Technical Considerations - -**Conforma CLI Execution** - -- Use `tokio::process::Command` for async execution to avoid blocking the runtime -- Stream stdout/stderr for real-time monitoring and logging -- Set execution timeouts (default: 5 minutes, configurable per policy) -- Handle large SBOM files efficiently by streaming to temporary files -- Capture exit codes for error handling (0=pass, 1=fail, 2+=error) -- Sanitize all CLI arguments to prevent injection attacks (use process args array, not shell strings) - -**Policy Management** - -- Support multiple policy types: Git repositories, OCI registries, local file paths -- Cache policy files locally to avoid repeated fetches from external sources -- Validate policy references before execution (check URL format, test connectivity) -- Track policy version/commit when storing validation results for reproducibility -- Support authentication for private policy repositories (tokens, SSH keys) - -**Result Storage** - -- Store structured violations in JSONB for efficient querying (filter by violation type, severity) -- Keep detailed reports in object storage (S3/Minio) to minimize database size -- Implement retention policies for old validation results (configurable, e.g., 90 days) -- Index frequently queried fields (sbom_id, status, executed_at) for performance -- Consider result aggregation for analytics dashboards - -**Error Handling** - -- Distinguish between validation failures (policy violations) and execution errors (CLI crashes) -- Provide actionable error messages to users (e.g., "Policy file not found at URL") -- Implement retry logic for transient failures (network timeouts, temporary service unavailability) -- Log all execution details for debugging (command, arguments, duration, exit code) -- Gracefully degrade when Conforma is unavailable (show cached results, queue for retry) - -**Security** - -- Validate and sanitize all user inputs (policy URLs, SBOM IDs) -- Restrict policy sources to trusted domains (configurable allowlist) -- Store authentication credentials securely (encrypted, never in logs) -- Implement proper authorization checks (only admins can modify policies) -- Audit log all validation executions and policy modifications - -**Performance** - -- Limit concurrent Conforma executions using semaphore (default: 5 concurrent) -- Implement queueing for validation requests during high load -- Cache policy files to reduce external network calls -- Use connection pooling for database operations -- Monitor execution times and resource usage for capacity planning -- Consider horizontal scaling for validation workloads - -## Consequences - -### Positive - -1. **Automated Policy Enforcement**: Organizations can automatically validate SBOMs without manual review -2. **Audit Trail**: Complete history of compliance checks stored in database -3. **Flexibility**: Support multiple policy configurations for different requirements -4. **Integration Ready**: REST API enables integration with CI/CD pipelines -5. **Scalability**: Async execution prevents blocking on long-running validations -6. **Extensibility**: Module design allows future enhancement (webhooks, notifications, etc.) -7. **Open Source**: Conforma is open-source and actively maintained - -### Trade-offs and Risks - -| Trade-off / Risk | Impact | Mitigation | -| ------------------------------- | ---------------------------------------- | -------------------------------------------------------------------------- | -| External CLI dependency | Requires Conforma installed on servers | Health checks, graceful error handling, retry logic | -| Process spawning overhead | Performance implications per validation | Async execution with configurable timeouts (default: 5 min) | -| Error handling complexity | CLI failures, timeouts, malformed output | Distinguish validation failures from execution errors; actionable messages | -| Version management | Conforma version compatibility | Document required version, validate on startup | -| Resource usage under load | Concurrent validations consume resources | Semaphore limits (default: 5), queueing, monitoring | -| No native API yet | CLI less efficient than REST integration | Adapter pattern for future API migration (see Phase 3) | -| Large SBOMs cause memory issues | Out-of-memory during validation | Stream SBOM to temp file, pass file path to Conforma | -| CLI injection attacks | Security vulnerability | Sanitize all inputs, use process args array (not shell strings) | -| Storage costs for reports | Growing storage over time | Retention policies, report compression | - -## Alternatives Considered - -### 1. In-Process Policy Engine - -**Pros**: No external dependencies, faster execution, native Rust integration - -**Cons**: Requires reimplementing Enterprise Contract logic, maintenance burden, divergence from upstream - -**Verdict**: Rejected - maintaining policy engine parity with EC would be significant effort - -### 2. Webhook-based Integration - -**Pros**: Decoupled, scalable, easier to manage separate service - -**Cons**: Additional infrastructure, network latency, complexity for simple use case - -**Verdict**: Deferred - could be future enhancements for large-scale deployments - -### 3. Embedded WASM Module - -**Pros**: Sandboxed, portable, no process spawning - -**Cons**: Conforma not available as WASM, would require major upstream changes - -**Verdict**: Rejected - not feasible with current Conforma implementation - -### 4. Batch Processing Queue - -**Pros**: Better resource management, retry logic, priority handling - -**Cons**: Adds complexity, requires queue infrastructure (Redis/RabbitMQ) - -**Verdict**: Deferred - implement if demand increases, start with simple async execution - -## References +### References - [Enterprise Contract (Conforma) GitHub](https://github.com/enterprise-contract/ec-cli) -- [Conforma CLI Documentation](https://github.com/enterprise-contract/ec-cli) - [Design Document](../design/enterprise-contract-integration.md) - [ADR-00005: Upload API for UI](./00005-ui-upload.md) - Similar async processing pattern - [ADR-00001: Graph Analytics](./00001-graph-analytics.md) - Database query patterns - -## Implementation Tracking - -### Backend Tasks - -- [ ] Create module structure under `modules/ec` -- [ ] Implement database migrations for new tables (`ec_policies`, `ec_validation_results`) -- [ ] Build Conforma CLI executor with async process handling -- [ ] Create result parser for JSON output -- [ ] Implement policy manager service -- [ ] Implement EC service orchestration layer -- [ ] Add REST API endpoints (validation, results, policies) -- [ ] Write unit and integration tests -- [ ] Add OpenAPI documentation -- [ ] Write deployment documentation (Conforma installation) -- [ ] Add monitoring and metrics (execution times, success rates) -- [ ] Implement error handling and retry logic -- [ ] Add authentication/authorization checks - -### Frontend Tasks - -- [ ] Add "Validate with EC" button to SBOM detail page -- [ ] Create compliance status badge component (Pass/Fail/Error) -- [ ] Implement validation results display with summary statistics -- [ ] Build violations list component with expandable details -- [ ] Create validation history timeline view -- [ ] Add policy reference management UI (admin pages) - - [ ] Policy reference list view with search/filter (shows name, external URL, type) - - [ ] Policy reference create/edit form (Git URL, OCI ref, auth config) - - [ ] Policy reference delete confirmation - - [ ] Test policy connectivity button (validate URL is reachable) -- [ ] Add report download functionality (JSON/HTML) -- [ ] Create detailed report preview modal -- [ ] Implement loading indicators for validation execution -- [ ] Add toast notifications for validation completion/errors -- [ ] Create compliance trend charts (optional, if analytics desired) -- [ ] Add help tooltips and documentation links -- [ ] Ensure responsive design for mobile/tablet views -- [ ] Add accessibility features (ARIA labels, keyboard navigation) - -## Open Questions - -1. **Policy Storage & Caching**: Should policies be pulled from external sources (OCI registry, Git repository) at runtime, or should Trustify cache them locally? Should we rely on Conforma's built-in caching or implement our own? How to handle cache invalidation and TTL for cached policies? - -2. **Trigger Mechanism**: Should the EC validation run automatically immediately upon SBOM upload, or will this be a manually triggered/scheduled process? Should users be able to configure auto-validation per SBOM or globally? - -3. **Data Granularity**: Do we store the full raw JSON output from Conforma, or only a summary (Pass/Fail) and a list of specific policy violations to save database space? What's the trade-off between storage cost and audit completeness? - -4. **Conforma Versioning**: Which version of the Conforma CLI are we targeting initially, and how will we handle updates to the policy engine? Should we support multiple Conforma versions concurrently or enforce a single version? - -5. **Policy Source Security**: Should we validate/verify Git repository signatures for policy sources? How to handle private repositories requiring authentication (tokens, SSH keys)? - -6. **Multi-tenancy**: How to isolate policy references per organization in shared Trustify deployments? Should each organization have separate policy namespaces? - -7. **Rate Limiting**: Should we limit validation requests per user/organization to prevent resource exhaustion? What are reasonable limits? - -8. **Notifications**: Should validation results trigger notifications (email, webhook, Slack)? Should notifications be configurable per policy or per SBOM? - -9. **Automatic Re-validation**: Should we re-validate SBOMs when policy definitions are updated in their external repositories? How to detect policy changes (polling, webhooks)? - -10. **UI/UX**: Where in the UI should compliance status be most prominently displayed? Should we show compliance badges on SBOM list views or only in detail views? - -11. **Default Policies**: Should there be system-wide default policy references that apply to all SBOMs unless overridden? How to handle precedence (user-level vs org-level vs system-level)? From cdf828d6f1df99442e00788ca413e2aca86fa333 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Feb 2026 10:24:33 +0100 Subject: [PATCH 08/77] Remove S3 from system Architecture --- docs/adrs/00014-enterprise-contract-integration.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 5036577b5..51f915016 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -110,13 +110,11 @@ C4Context System(trustify, "Trustify", "RHTPA") System_Ext(conforma, "Conforma", "Enterprise Contract policy validation tool") - System_Ext(s3, "S3", "S3/Minio Storage") System_Ext(policyRepo, "Policy Repository", "Git repository or storage containing EC policies") Rel(user, trustify, "Request Compliance
View compliance status", "API/GUI") Rel(trustify, conforma, "Executes policy validation", "Spawn Process") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") - Rel(trustify, s3, "Stores reports", "S3 API") UpdateRelStyle(trustify, conforma, $offsetX="-40") UpdateRelStyle(user, trustify, $offsetX="-50", $offsetY="20") From 0f0714939c2e032f09937d73016bbe0754ae1464 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Feb 2026 10:26:17 +0100 Subject: [PATCH 09/77] Offset link --- docs/adrs/00014-enterprise-contract-integration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 51f915016..21de73a02 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -108,7 +108,6 @@ C4Context Person(user, "Trustify User", "Analyst validating SBOMs") System(trustify, "Trustify", "RHTPA") - System_Ext(conforma, "Conforma", "Enterprise Contract policy validation tool") System_Ext(policyRepo, "Policy Repository", "Git repository or storage containing EC policies") @@ -116,8 +115,9 @@ C4Context Rel(trustify, conforma, "Executes policy validation", "Spawn Process") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") - UpdateRelStyle(trustify, conforma, $offsetX="-40") UpdateRelStyle(user, trustify, $offsetX="-50", $offsetY="20") + UpdateRelStyle(trustify, conforma, $offsetX="-40") + UpdateRelStyle(conforma, policyRepo,$offsetX="-40") UpdateLayoutConfig($c4ShapeInRow="2", $c4BoundaryInRow="1") ``` From e9183f57ab3930b13dba1629a937eef3adf1ea19 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Feb 2026 10:29:29 +0100 Subject: [PATCH 10/77] Component diagram: standalone policy system --- docs/adrs/00014-enterprise-contract-integration.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 21de73a02..e0ef90d88 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -142,6 +142,9 @@ C4Container Container_Boundary(conforma, "Conforma System") { System_Ext(conforma, "Conforma CLI", "External policy validation tool") + } + + Container_Boundary(policyRepo, "Policy System") { System_Ext(policyRepo, "Policy Repository", "Git repository with EC policies") } @@ -161,7 +164,7 @@ C4Container UpdateRelStyle(ecModule, postgres, $offsetX="-40", $offsetY="10") UpdateRelStyle(storage, s3, $offsetX="-40", $offsetY="10") - UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="3") + UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` ### Container Diagram From 08d40fad6dfb0fba0982a2f68a7c3e325139498e Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Feb 2026 11:29:06 +0100 Subject: [PATCH 11/77] Component details --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index e0ef90d88..88ea02ca0 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -180,7 +180,7 @@ C4Component Component(conformaExecutor, "Conforma Executor", "Async Process", "Spawns Conforma CLI and captures output") Component(ecService, "EC Service", "Business logic", "Orchestrates validation workflow") Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output into structured data") - Component(policyManager, "Policy Manager", "Configuration", "Manages EC policy references") + Component(policyManager, "Policy Manager", "Business logic", "Manages EC policy references and configuration") Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results") } From aa9bded72c6103fd2176956707c2272eb9abe4e8 Mon Sep 17 00:00:00 2001 From: arpcv Date: Mon, 23 Feb 2026 14:57:02 +0100 Subject: [PATCH 12/77] Refine SBOM validation process details and update terminology in enterprise contract integration documentation --- .../00014-enterprise-contract-integration.md | 23 ++++++++----------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 88ea02ca0..c63c03386 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -16,7 +16,7 @@ Enterprise Contract (Conforma) is an open-source policy enforcement tool activel Users need the ability to: -1. Automatically validate SBOMs against organizational policies +1. Validate SBOMs against organizational policies 2. Define and manage multiple policy configurations 3. View compliance status and violation details for each SBOM 4. Track compliance history over time @@ -29,6 +29,13 @@ We will integrate Conforma into Trustify as an optional validation service by sp Validation is manually triggered — not automatic on SBOM upload. Validation on upload is deferred to a follow-up version. +Uploaded SBOMs start in "Pending" status and are not discoverable until validated. EC validation is one mechanism by which an SBOM can move from "Pending" to "Accepted" or "Rejected": + +- An SBOM in "Pending" can be submitted for EC validation. +- A passing EC result transitions the SBOM to "Accepted". +- A failing EC result transitions it to "Rejected", with the violation details linked. +- An EC execution error (CLI crash, policy fetch failure) does not change the SBOM's policy status — the SBOM stays Pending and the error is surfaced separately, so it doesn't silently block an SBOM. + What is stored where - PostgreSQL: validation status, structured violations (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, status, executed_at. @@ -37,18 +44,6 @@ What is stored where Storing full JSON in S3 rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB violations JSONB holds enough structure for filtering and dashboards without duplicating the full payload. -### Deferred Validation - -I has been decided that uploaded SBOMs start in "Pending" status and are not discoverable until validated. -EC validation is one mechanism by which an SBOM can move from "Pending" to "Accepted" or "Rejected". - -Concretely: - - An SBOM in "Pending" can be submitted for EC validation. - A passing EC result transitions the SBOM to "Accepted". - A failing EC result transitions it to "Rejected", with the violation details linked. - An EC execution error (CLI crash, policy fetch failure) does not change the SBOM's policy status — the SBOM stays Pending and the error is surfaced separately, so it doesn't silently block an SBOM. - ## Consequences Integrating via CLI spawning rather than a native API introduces an external process dependency that adds operational overhead (Conforma must be installed and version-pinned on every server) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma API exists yet. The executor is built behind an adapter interface so the implementation can be swapped for a REST client in Phase 3 without changes to the service layer or API. @@ -271,7 +266,7 @@ sequenceDiagram ### The Data Model -**`ec_policies`** - Stores references to external policies, not the policies themselves +**`ec_policy`** - Stores references to external policies, not the policies themselves - `id` (UUID, PK) - `name` (VARCHAR, unique) - User-friendly name label From ce254988044c5006364b19df3078a21709ed0189 Mon Sep 17 00:00:00 2001 From: arpcv Date: Mon, 23 Feb 2026 15:03:02 +0100 Subject: [PATCH 13/77] Integrated the EC result status; Fix plural for db table; Remove irrelevant requirement --- docs/adrs/00014-enterprise-contract-integration.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index c63c03386..6e676a9ae 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -329,10 +329,6 @@ Policy version/commit hash is recorded in the conforma_version field of each res Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. -#### Data Retention - -Validation results accumulate over time. A retention policy (default: 90 days, configurable) should be implemented to prune old ec_validation_results rows and their corresponding S3 reports. This is deferred to the implementation phase but must be included before production rollout. - ### Module Structure ``` From 09bb93690e985d5b7825d762bd0fc63197a8bb27 Mon Sep 17 00:00:00 2001 From: arpcv Date: Wed, 25 Feb 2026 16:32:41 +0100 Subject: [PATCH 14/77] External HTTP API/Webhook Wraper --- .../00014-enterprise-contract-integration.md | 71 ++++++++----------- 1 file changed, 31 insertions(+), 40 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 6e676a9ae..b1d7333be 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -25,10 +25,13 @@ Users need the ability to: ## Decision -We will integrate Conforma into Trustify as an optional validation service by spawning the Conforma CLI asynchronously. +We will integrate Conforma into Trustify as a user triggered validation service by interacting with Conforma CLI. Validation is manually triggered — not automatic on SBOM upload. Validation on upload is deferred to a follow-up version. +Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. +A HTTP wrapper will act as a proxy between Trustify EC service and Conforma CLI. + Uploaded SBOMs start in "Pending" status and are not discoverable until validated. EC validation is one mechanism by which an SBOM can move from "Pending" to "Accepted" or "Rejected": - An SBOM in "Pending" can be submitted for EC validation. @@ -46,29 +49,11 @@ Storing full JSON in S3 rather than only a summary was chosen explicitly to pres ## Consequences -Integrating via CLI spawning rather than a native API introduces an external process dependency that adds operational overhead (Conforma must be installed and version-pinned on every server) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma API exists yet. The executor is built behind an adapter interface so the implementation can be swapped for a REST client in Phase 3 without changes to the service layer or API. +Using a HTTP API wrapper decouples the validation process into a external service. +This will better catter for large-scale deployments as EC validation has its own constraint. +Meanwhile it adds infrastructure complexity as the Webhook will need to be deployed alongside the EC system -### Trade-off/Risk & Mitigation - -- Conforma must be installed on servers - - Health check on startup - - graceful degradation showing cached results -- Process spawn overhead per validation - - Async execution; configurable timeout (default 5 min) -- CLI injection attacks - - Args array only, never shell strings; all user inputs sanitized -- Version compatibility - - Document required Conforma version; validate on startup -- Concurrent load exhausting resources - - Semaphore (default: 5) - - 429 on exhaustion -- No native API yet - - Adapter pattern for future migration (Phase 3) -- Large SBOMs causing OOM - - Stream to temp file - - pass path to Conforma -- Growing S3 storage costs - - Retention policy (90-day default) +Integrating via CLI spawning rather than a native API introduces an external process dependency that adds operational overhead (Conforma must be installed and version-pinned on every server) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma API exists yet. The executor is built behind an adapter interface so the implementation can be swapped for a REST client in Phase 3 without changes to the service layer or API. ### Alternatives Considered @@ -76,9 +61,9 @@ Integrating via CLI spawning rather than a native API introduces an external pro Reimplementing Enterprise Contract logic in Rust would diverge from upstream and create significant maintenance burden. -#### Webhook-based Integration: Deferred +#### Direct Integration: Rejected -Decouples validation into a separate service, which is better for large-scale deployments but adds infrastructure complexity premature for initial scope. +Couple validation integrated within Trustify service through a directly controlled component was simpler but worse for large-scale deployments. #### Embedded WASM Module: Rejected @@ -168,19 +153,23 @@ C4Container C4Component title EC Validation Module - Component Diagram - Container(api, "API Gateway", "Actix-web", "REST API for EC operations") - - Container_Boundary(ecModule, "EC Validation Module") { - Component(ecEndpoints, "EC Endpoints", "Actix-web handlers", "REST endpoints for validation operations") - Component(conformaExecutor, "Conforma Executor", "Async Process", "Spawns Conforma CLI and captures output") - Component(ecService, "EC Service", "Business logic", "Orchestrates validation workflow") - Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output into structured data") - Component(policyManager, "Policy Manager", "Business logic", "Manages EC policy references and configuration") - Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results") + Deployment_Node(trustifySystem, "Trustify System") { + Container(api, "API Gateway", "Actix-web", "REST API for EC operations") + Container_Boundary(ecModule, "EC Validation Module") { + Container(ecEndpoints, "EC Endpoints", "Actix-web handlers", "REST endpoints for validation operations") + Component(ecService, "EC Service", "Business logic", "Orchestrates validation workflow") + Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output into structured data") + Component(policyManager, "Policy Manager", "Business logic", "Manages EC policy references and
configuration") + Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results") + } } - - Container_Boundary(external, "External System") { - System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") + Deployment_Node(external, "External System") { + Deployment_Node(trustifyPod, "Trustify Pod") { + Component(conformaECWrapper, "EC Wrapper", "HTTP API/Webhook") + } + Deployment_Node(conformaPod, "Conforma Pod") { + System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") + } } Container_Boundary(dbms, "Database") { @@ -194,17 +183,19 @@ C4Component Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT ec_policies", "SQL") - Rel(ecService, conformaExecutor, "request_validation()", "Function call") - Rel(conformaExecutor, conforma, "Runs CLI command", "Process spawn") + Rel(ecService, conformaECWrapper, "POST /api/v1/validation", "HTTP") + Rel(conformaECWrapper, conforma, "ec validate", "Process spawn") Rel(ecService, resultParser, "parse_output()", "Function call") Rel(ecService, resultPersistence, "save_results()", "Function call") Rel(resultPersistence, postgres, "INSERT ec_validation_results", "SQL") Rel(ecService, s3, "Store EC report", "S3 API") UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") + UpdateRelStyle(ecEndpoints, ecService, $offsetX="-60", $offsetY="+40") + UpdateRelStyle(ecService, resultParser, $offsetX="-60", $offsetY="+0") + UpdateRelStyle(ecService, resultPersistence, $offsetX="-60", $offsetY="+80") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") - ``` ### The main sequence Diagram From 451a82347ee998f8af47efe66fb04b768cd2ba8d Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 3 Mar 2026 13:56:18 +0100 Subject: [PATCH 15/77] Adjust styles for components and containers --- .../00014-enterprise-contract-integration.md | 24 ++++++++++++------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index b1d7333be..41920406c 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -91,7 +91,7 @@ C4Context System_Ext(conforma, "Conforma", "Enterprise Contract policy validation tool") System_Ext(policyRepo, "Policy Repository", "Git repository or storage containing EC policies") - Rel(user, trustify, "Request Compliance
View compliance status", "API/GUI") + Rel(user, trustify, "Request compliance
View compliance status", "API/GUI") Rel(trustify, conforma, "Executes policy validation", "Spawn Process") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") @@ -112,7 +112,7 @@ C4Container Container_Boundary(trustify, "Trustify System") { - Container(webui, "Web UI", "React/TypeScript", "Trustify GUI") + Container(webui, "Web UI", "Rust/Actix", "Trustify GUI") Container(api, "API Gateway", "Actix-web", "REST API endpoints for SBOM
and compliance operations") ContainerDb(postgres, "PostgreSQL", "DBMS", "Stores SBOM metadata, relationships,
and EC validation results") Container(ecModule, "EC Validation Module", "Rust", "Orchestrates Conforma CLI
execution and result persistence") @@ -121,6 +121,7 @@ C4Container } Container_Boundary(conforma, "Conforma System") { + Container(ecWrapper, "EC Wrapper", "Rust/Actix", "HTTP Wrapper") System_Ext(conforma, "Conforma CLI", "External policy validation tool") } @@ -128,11 +129,12 @@ C4Container System_Ext(policyRepo, "Policy Repository", "Git repository with EC policies") } - Rel(user, webui, "Views compliance status", "HTTPS") - Rel(user, api, "Views compliance status", "RESTful") - Rel(webui, api, "API calls", "JSON/HTTPS") + Rel(user, webui, "Views compliance status", "HTTP API") + Rel(user, api, "Views compliance status", "HTTP API") + Rel(webui, api, "API calls", "JSON/HTTP API") Rel(api, ecModule, "Triggers validation", "Function call") - Rel(ecModule, conforma, "Executes validation", "CLI/Process") + Rel(ecModule, ecWrapper, "POST /validate {SBOM} {policy}", "HTTP API") + Rel(ecWrapper, conforma, "ec validate input {SBOM} {policy}", "Spawned command") Rel(ecModule, postgres, "Saves validation
results", "SQL") Rel(ecModule, storage, "Stores EC reports", "Function call") Rel(storage, s3, "Persists reports", "S3 API") @@ -141,6 +143,7 @@ C4Container UpdateRelStyle(user, webui, $offsetX="-60", $offsetY="30") UpdateRelStyle(user, api, $offsetX="-60", $offsetY="-50") UpdateRelStyle(webui, api, $offsetX="-40", $offsetY="10") + UpdateRelStyle(ecModule, ecWrapper, $offsetX="-50", $offsetY="-20") UpdateRelStyle(ecModule, postgres, $offsetX="-40", $offsetY="10") UpdateRelStyle(storage, s3, $offsetX="-40", $offsetY="10") @@ -165,7 +168,7 @@ C4Component } Deployment_Node(external, "External System") { Deployment_Node(trustifyPod, "Trustify Pod") { - Component(conformaECWrapper, "EC Wrapper", "HTTP API/Webhook") + Component(ecWrapper, "EC Wrapper", "Actix-web handlers", "HTTP API/Webhook") } Deployment_Node(conformaPod, "Conforma Pod") { System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") @@ -183,8 +186,9 @@ C4Component Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT ec_policies", "SQL") - Rel(ecService, conformaECWrapper, "POST /api/v1/validation", "HTTP") - Rel(conformaECWrapper, conforma, "ec validate", "Process spawn") + Rel(ecService, ecWrapper, "POST /api/v1/validation", "HTTP + callback URL") + Rel(ecWrapper, conforma, "ec validate", "Process spawn") + Rel(ecWrapper, ecEndpoints, "POST /validate/job/{id}", "JSON/HTTPS") Rel(ecService, resultParser, "parse_output()", "Function call") Rel(ecService, resultPersistence, "save_results()", "Function call") Rel(resultPersistence, postgres, "INSERT ec_validation_results", "SQL") @@ -192,6 +196,8 @@ C4Component UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") UpdateRelStyle(ecEndpoints, ecService, $offsetX="-60", $offsetY="+40") + UpdateRelStyle(ecService, ecWrapper, $offsetX="-20", $offsetY="10") + UpdateRelStyle(ecWrapper, ecEndpoints, $offsetX="20", $offsetY="-40") UpdateRelStyle(ecService, resultParser, $offsetX="-60", $offsetY="+0") UpdateRelStyle(ecService, resultPersistence, $offsetX="-60", $offsetY="+80") From cf45b4563c8925b14ee36e8ab33d7581921c67c6 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 3 Mar 2026 14:58:13 +0100 Subject: [PATCH 16/77] Swap titles --- docs/adrs/00014-enterprise-contract-integration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 41920406c..88721bcba 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -102,7 +102,7 @@ C4Context UpdateLayoutConfig($c4ShapeInRow="2", $c4BoundaryInRow="1") ``` -### Component Diagram - EC Validation Module +### Container Diagram - EC Validation Module ```mermaid C4Container @@ -150,7 +150,7 @@ C4Container UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` -### Container Diagram +### Component Diagram ```mermaid C4Component From 4783079c0e13043233b3bbd62fa6702f2e021382 Mon Sep 17 00:00:00 2001 From: gildub Date: Tue, 3 Mar 2026 19:09:17 +0100 Subject: [PATCH 17/77] Conforma JSON output with HTTP Wrapper feed back --- .../00014-enterprise-contract-integration.md | 204 +++++++++++++----- 1 file changed, 148 insertions(+), 56 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 88721bcba..22062fe03 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -32,12 +32,15 @@ Validation on upload is deferred to a follow-up version. Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. A HTTP wrapper will act as a proxy between Trustify EC service and Conforma CLI. -Uploaded SBOMs start in "Pending" status and are not discoverable until validated. EC validation is one mechanism by which an SBOM can move from "Pending" to "Accepted" or "Rejected": +Each SBOM + policy pair has a validation state that follows this lifecycle: -- An SBOM in "Pending" can be submitted for EC validation. -- A passing EC result transitions the SBOM to "Accepted". -- A failing EC result transitions it to "Rejected", with the violation details linked. -- An EC execution error (CLI crash, policy fetch failure) does not change the SBOM's policy status — the SBOM stays Pending and the error is surfaced separately, so it doesn't silently block an SBOM. +- **Pending** — default state when an SBOM is uploaded. No validation has been triggered yet for this SBOM against this policy. +- **In Progress** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. +- **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. +- **Fail** — Conforma validation found policy violations; violation details are linked. +- **Error** — an execution error occurred (CLI crash, policy fetch failure, timeout). The error is surfaced separately, and the validation can be re-triggered. + +The "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. What is stored where @@ -133,7 +136,8 @@ C4Container Rel(user, api, "Views compliance status", "HTTP API") Rel(webui, api, "API calls", "JSON/HTTP API") Rel(api, ecModule, "Triggers validation", "Function call") - Rel(ecModule, ecWrapper, "POST /validate {SBOM} {policy}", "HTTP API") + Rel(ecModule, ecWrapper, "POST /api/v1/validation {SBOM, policy_ref}
← returns validation_id", "HTTP API") + Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "HTTP callback") Rel(ecWrapper, conforma, "ec validate input {SBOM} {policy}", "Spawned command") Rel(ecModule, postgres, "Saves validation
results", "SQL") Rel(ecModule, storage, "Stores EC reports", "Function call") @@ -186,9 +190,9 @@ C4Component Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT ec_policies", "SQL") - Rel(ecService, ecWrapper, "POST /api/v1/validation", "HTTP + callback URL") + Rel(ecService, ecWrapper, "POST /api/v1/validation → returns validation_id", "HTTP") Rel(ecWrapper, conforma, "ec validate", "Process spawn") - Rel(ecWrapper, ecEndpoints, "POST /validate/job/{id}", "JSON/HTTPS") + Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") Rel(ecService, resultParser, "parse_output()", "Function call") Rel(ecService, resultPersistence, "save_results()", "Function call") Rel(resultPersistence, postgres, "INSERT ec_validation_results", "SQL") @@ -197,68 +201,108 @@ C4Component UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") UpdateRelStyle(ecEndpoints, ecService, $offsetX="-60", $offsetY="+40") UpdateRelStyle(ecService, ecWrapper, $offsetX="-20", $offsetY="10") - UpdateRelStyle(ecWrapper, ecEndpoints, $offsetX="20", $offsetY="-40") + UpdateRelStyle(ecWrapper, api, $offsetX="20", $offsetY="-40") UpdateRelStyle(ecService, resultParser, $offsetX="-60", $offsetY="+0") UpdateRelStyle(ecService, resultPersistence, $offsetX="-60", $offsetY="+80") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` -### The main sequence Diagram +### Sequence Diagram — User Request (synchronous) ```mermaid sequenceDiagram autonumber actor User participant API as Trustify API + participant EP as EC Endpoints participant VS as EC Service participant PM as Policy Manager participant DB as PostgreSQL participant S3 as Object Storage - participant Conf as Conforma CLI + participant Wrapper as EC Wrapper (HTTP) User->>API: POST /api/v2/sbom/{sbom_id}/ec-validate {policy_id} + API->>EP: dispatch request + EP->>VS: validate_sbom(sbom_id, policy_id) VS->>PM: get_policy_configuration(policy_id) PM->>DB: SELECT * FROM ec_policies WHERE id = ? alt Policy not found PM-->>VS: Error: PolicyNotFound - VS-->>API: 404 Not Found + VS-->>EP: 404 Not Found + EP-->>API: 404 Not Found end PM-->>VS: PolicyConfig {name, policy_ref, type} VS->>DB: SELECT * FROM sbom WHERE id = ? alt SBOM not found - VS-->>API: 404 Not Found + VS-->>EP: 404 Not Found + EP-->>API: 404 Not Found end + + VS->>DB: SELECT * FROM ec_validation_results WHERE sbom_id = ? AND policy_id = ? AND status = 'in_progress' + alt Validation already in progress + VS-->>EP: 409 Conflict {existing job_id} + EP-->>API: 409 Conflict + API-->>User: 409 Conflict — validation already in progress + end + VS->>S3: retrieve_sbom_document(sbom_id) S3-->>VS: SBOM document (JSON/XML) - VS->>VS: Write SBOM to temp file - VS->>Conf: conforma validate input --file "$SBOM_PATH" --policy --output json --show-successes --info + VS->>Wrapper: POST /api/v1/validation {SBOM, policy_ref} + Wrapper-->>VS: 202 Accepted {validation_id} + + VS->>DB: INSERT ec_validation_results (status='in_progress', sbom_id, policy_id, validation_id, ...) + VS-->>EP: 202 Accepted {validation_id} + EP-->>API: 202 Accepted {validation_id} + API-->>User: 202 Accepted {validation_id} +``` + +### Sequence Diagram — Async Validation Processing + +```mermaid +sequenceDiagram + autonumber + participant API as Trustify API + participant EP as EC Endpoints + participant VS as EC Service + participant DB as PostgreSQL + participant S3 as Object Storage + participant Wrapper as EC Wrapper (HTTP) + participant Conf as Conforma CLI + + Note over Wrapper: Wrapper holds validation_id from the initial request + + Wrapper->>Wrapper: Write SBOM to temp file + Wrapper->>Conf: ec validate input --file "$SBOM_PATH" --policy --output json --show-successes --info alt Exit code 0 — pass - Conf-->>VS: {result: "PASS", violations: []} - VS->>DB: INSERT ec_validation_results (status='pass', violations=[], ...) + Conf-->>Wrapper: {result: "PASS", violations: []} + else Exit code 1 — fail (policy violations) + Conf-->>Wrapper: {result: "FAIL", violations: [{rule, severity, message}]} + else Exit code 2+ — execution error + Conf-->>Wrapper: stderr: "Policy file not found" + end + + Wrapper->>Wrapper: Cleanup temp files + Wrapper->>API: POST /api/v2/ec/validation/{validation_id}/result {conforma JSON output} + API->>EP: dispatch callback + EP->>VS: process_validation_result(validation_id, result) + + alt Pass + VS->>DB: UPDATE ec_validation_results SET status='pass', violations=[] VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_url = ? - VS->>DB: UPDATE sbom SET policy_status = 'Accepted' - VS-->>API: 200 {passed: true, violations: 0, report_url} - else Exit code 1 — fail (policy violations) - Conf-->>VS: {result: "FAIL", violations: [{rule, severity, message}]} - VS->>DB: INSERT ec_validation_results (status='fail', violations=json, ...) + else Fail + VS->>DB: UPDATE ec_validation_results SET status='fail', violations=json VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_url = ? - VS->>DB: UPDATE sbom SET policy_status = 'Rejected' - VS-->>API: 200 {passed: false, violations: [...], report_url} - else Exit code 2+ — execution error - Conf-->>VS: stderr: "Policy file not found" - VS->>DB: INSERT ec_validation_results (status='error', error_message=stderr) - Note over VS,DB: SBOM policy_status unchanged — stays Pending - VS-->>API: 500 {error: "Validation failed to execute", detail: stderr} + else Error + VS->>DB: UPDATE ec_validation_results SET status='error', error_message=detail + Note over VS,DB: Validation can be re-triggered (new request will create a new row with status='in_progress') end - - VS->>VS: Cleanup temp files ``` ### The Data Model @@ -276,9 +320,10 @@ sequenceDiagram **`ec_validation_results`** - one row per validation execution - `id` (UUID, PK) +- `validation_id` (VARCHAR, unique) - ID returned by the EC Wrapper, used for callback correlation - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → ec_policies) -- `status` (VARCHAR) - 'pass', 'fail', 'error' +- `status` (VARCHAR) - 'pending', 'in_progress', 'pass', 'fail', 'error' - `violations` (JSONB) - Structured violation data for querying - `summary` (JSONB) - Total checks, passed, failed, warnings - `report_url` (VARCHAR) - S3 URL to detailed report @@ -290,10 +335,11 @@ sequenceDiagram ### API Endpoints ``` -POST /api/v2/sboms/{id}/ec-validate # Trigger validation -GET /api/v2/sboms/{id}/ec-report # Get latest validation result -GET /api/v2/sboms/{id}/ec-report/history # Get validation history -GET /api/v2/ec/report/{result_id} # Download detailed report from S3 +POST /api/v2/sboms/{id}/ec-validate # Trigger validation +GET /api/v2/sboms/{id}/ec-report # Get latest validation result +GET /api/v2/sboms/{id}/ec-report/history # Get validation history +GET /api/v2/ec/report/{result_id} # Download detailed report from S3 +POST /api/v2/ec/validation/{validation_id}/result # Callback: EC Wrapper posts Conforma result POST /api/v2/ec/policies # Create policy reference (admin) GET /api/v2/ec/policies # List policy references @@ -302,6 +348,28 @@ PUT /api/v2/ec/policies/{id} # Update policy reference (admin) DELETE /api/v2/ec/policies/{id} # Delete policy reference (admin) ``` +### Module Structure + +``` +modules/ec/ +├── Cargo.toml +└── src/ + ├── lib.rs + ├── endpoints/ + │ └── mod.rs # REST endpoints + ├── model/ + │ ├── mod.rs + │ ├── policy.rs # Policy API models + │ └── validation.rs # Validation result models + ├── service/ + │ ├── mod.rs + │ ├── ec_service.rs # Main orchestration + │ ├── policy_manager.rs # Policy configuration + │ ├── executor.rs # Conforma CLI execution + │ └── result_parser.rs # Output parsing + └── error.rs # Error types +``` + ### Technical Considerations #### Conforma CLI Execution @@ -326,26 +394,50 @@ Policy version/commit hash is recorded in the conforma_version field of each res Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. -### Module Structure - -``` -modules/ec/ -├── Cargo.toml -└── src/ - ├── lib.rs - ├── endpoints/ - │ └── mod.rs # REST endpoints - ├── model/ - │ ├── mod.rs - │ ├── policy.rs # Policy API models - │ └── validation.rs # Validation result models - ├── service/ - │ ├── mod.rs - │ ├── ec_service.rs # Main orchestration - │ ├── policy_manager.rs # Policy configuration - │ ├── executor.rs # Conforma CLI execution - │ └── result_parser.rs # Output parsing - └── error.rs # Error types +### Structure of JSON returned from Conformat CLI validation request (from an example) + +```json +{ + "success": false, + "filepaths": [ + { + "filepath": "sboms/registry.redhat.io__rhtas__ec-rhel9__sha256__ea49a30eef5a2948b04540666a12048dd082625ce9f755acd3ece085c7d7937e.json", + "violations": [ + { + "msg": "There are 2942 packages which is more than the permitted maximum of 510.", + "metadata": { + "code": "hello_world.minimal_packages", + "description": "Just an example... To exclude this rule add \"hello_world.minimal_packages\" to the `exclude` section of the policy configuration.", + "solution": "You need to reduce the number of dependencies in this artifact.", + "title": "Check we don't have too many packages" + } + } + ], + "warnings": [], + "successes": [ + { + "msg": "Pass", + "metadata": { + "code": "hello_world.valid_spdxid", + "description": "Make sure that the SPDXID value found in the SBOM matches a list of allowed values.", + "title": "Check for valid SPDXID value" + } + } + ], + "success": false, + "success-count": 1 + } + ], + "policy": { + "sources": [ + { + "policy": ["github.com/conforma/policy//policy/lib", "./policy"] + } + ] + }, + "ec-version": "v0.8.83", + "effective-time": "2026-03-03T14:36:55.807826709Z" +} ``` ### References From 7ae8b1ca0c90c54abff11fad699a53c3fb955144 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 3 Mar 2026 20:32:04 +0100 Subject: [PATCH 18/77] Fix inconsitencies --- .../00014-enterprise-contract-integration.md | 39 +++++++++---------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 22062fe03..6847a6fb5 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -30,11 +30,11 @@ Validation is manually triggered — not automatic on SBOM upload. Validation on upload is deferred to a follow-up version. Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. -A HTTP wrapper will act as a proxy between Trustify EC service and Conforma CLI. +An EC Wrapper (HTTP service) acts as a proxy between Trustify's EC service and Conforma CLI. Each SBOM + policy pair has a validation state that follows this lifecycle: -- **Pending** — default state when an SBOM is uploaded. No validation has been triggered yet for this SBOM against this policy. +- **Pending** — initial state, set when an SBOM is associated with a policy (e.g., a default policy assigned at SBOM upload time; upload itself is outside the scope of this ADR). Indicates no validation has been triggered yet for this SBOM against this policy. - **In Progress** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. - **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. - **Fail** — Conforma validation found policy violations; violation details are linked. @@ -52,11 +52,9 @@ Storing full JSON in S3 rather than only a summary was chosen explicitly to pres ## Consequences -Using a HTTP API wrapper decouples the validation process into a external service. -This will better catter for large-scale deployments as EC validation has its own constraint. -Meanwhile it adds infrastructure complexity as the Webhook will need to be deployed alongside the EC system +Using an EC Wrapper decouples the validation process into an external service. This better caters for large-scale deployments as EC validation has its own resource constraints. Meanwhile it adds infrastructure complexity as the EC Wrapper must be deployed and maintained alongside the Conforma CLI. -Integrating via CLI spawning rather than a native API introduces an external process dependency that adds operational overhead (Conforma must be installed and version-pinned on every server) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma API exists yet. The executor is built behind an adapter interface so the implementation can be swapped for a REST client in Phase 3 without changes to the service layer or API. +Within the EC Wrapper, Conforma is invoked via CLI spawning rather than a native API. This introduces an operational dependency (Conforma must be installed and version-pinned on every EC Wrapper instance) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma REST API exists yet. On the Trustify side, the EC service interacts with the EC Wrapper over HTTP and is built behind an adapter interface, so the implementation can be swapped for a direct Conforma REST client when one becomes available, without changes to the service layer or API. ### Alternatives Considered @@ -78,7 +76,7 @@ A Redis/RabbitMQ queue would improve retry handling and priority management; imp ### Future API Migration -When Conforma provides a REST API, the executor.rs adapter is replaced with an HTTP client. A feature flag (ec-api-mode) allows gradual migration. No changes to service layer, API endpoints, or UI are required. +When Conforma provides a REST API, the EC Wrapper can be replaced by pointing Trustify's adapter directly at the Conforma REST endpoint. A feature flag (`ec-api-mode`) allows gradual migration. No changes to the service layer, API endpoints, or UI are required. ## The solution @@ -95,7 +93,7 @@ C4Context System_Ext(policyRepo, "Policy Repository", "Git repository or storage containing EC policies") Rel(user, trustify, "Request compliance
View compliance status", "API/GUI") - Rel(trustify, conforma, "Executes policy validation", "Spawn Process") + Rel(trustify, conforma, "Triggers policy validation", "HTTP API") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") UpdateRelStyle(user, trustify, $offsetX="-50", $offsetY="20") @@ -118,17 +116,17 @@ C4Container Container(webui, "Web UI", "Rust/Actix", "Trustify GUI") Container(api, "API Gateway", "Actix-web", "REST API endpoints for SBOM
and compliance operations") ContainerDb(postgres, "PostgreSQL", "DBMS", "Stores SBOM metadata, relationships,
and EC validation results") - Container(ecModule, "EC Validation Module", "Rust", "Orchestrates Conforma CLI
execution and result persistence") + Container(ecModule, "EC Validation Module", "Rust", "Orchestrates validation via
EC Wrapper and persists results") ContainerDb(s3, "Object Storage", "S3/Minio", "Stores SBOM documents and EC reports") Container(storage, "Storage Service", "Rust", "Manages document storage
(SBOMs, policy results)") } - Container_Boundary(conforma, "Conforma System") { + Container_Boundary(conformaSystem, "Conforma System") { Container(ecWrapper, "EC Wrapper", "Rust/Actix", "HTTP Wrapper") System_Ext(conforma, "Conforma CLI", "External policy validation tool") } - Container_Boundary(policyRepo, "Policy System") { + Container_Boundary(policySystem, "Policy System") { System_Ext(policyRepo, "Policy Repository", "Git repository with EC policies") } @@ -172,7 +170,7 @@ C4Component } Deployment_Node(external, "External System") { Deployment_Node(trustifyPod, "Trustify Pod") { - Component(ecWrapper, "EC Wrapper", "Actix-web handlers", "HTTP API/Webhook") + Component(ecWrapper, "EC Wrapper", "Actix-web handlers", "HTTP API") } Deployment_Node(conformaPod, "Conforma Pod") { System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") @@ -307,7 +305,7 @@ sequenceDiagram ### The Data Model -**`ec_policy`** - Stores references to external policies, not the policies themselves +**`ec_policies`** - Stores references to external policies, not the policies themselves - `id` (UUID, PK) - `name` (VARCHAR, unique) - User-friendly name label @@ -329,7 +327,8 @@ sequenceDiagram - `report_url` (VARCHAR) - S3 URL to detailed report - `executed_at` (TIMESTAMP) - `execution_duration_ms` (INTEGER) -- `conforma_version` (VARCHAR) - For reproducibility +- `conforma_version` (VARCHAR) - Conforma CLI version used (e.g., `v0.8.83`), for reproducibility +- `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status ### API Endpoints @@ -365,16 +364,16 @@ modules/ec/ │ ├── mod.rs │ ├── ec_service.rs # Main orchestration │ ├── policy_manager.rs # Policy configuration - │ ├── executor.rs # Conforma CLI execution + │ ├── executor.rs # EC Wrapper HTTP client (adapter) │ └── result_parser.rs # Output parsing └── error.rs # Error types ``` ### Technical Considerations -#### Conforma CLI Execution +#### Conforma CLI Execution (EC Wrapper) -Conforma is invoked via tokio::process::Command to avoid blocking the async runtime. All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are streamed to a temp file and passed by path rather than piped via stdin, which avoids OOM issues with the parent process. +The EC Wrapper invokes Conforma via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are written to a temp file and passed by path rather than piped via stdin, which avoids OOM issues. Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected failure, not an error), 2+ = execution error. It is important to distinguish 1 from 2+ in error handling — a policy violation is a valid result that should be surfaced to the user, not treated as a system failure. @@ -382,19 +381,19 @@ Temp files (SBOM, any cached policy material) are cleaned up in a finally-equiva #### Concurrency and Backpressure -Concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, incoming validation requests return 429 Too Many Requests immediately rather than queuing or blocking indefinitely. This makes the capacity limit explicit to callers (e.g. CI pipelines can implement their own retry with backoff). If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. +On the EC Wrapper side, concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the EC Wrapper returns 429 Too Many Requests to Trustify, which propagates the status to the caller. This makes the capacity limit explicit to callers (e.g., CI pipelines can implement their own retry with backoff). On the Trustify side, the "In Progress" concurrency guard (409 Conflict) prevents duplicate validation runs for the same SBOM + policy pair. If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. #### Policy Management ec_policies stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and must be encrypted at rest; they are never logged. -Policy version/commit hash is recorded in the conforma_version field of each result row, enabling reproducibility and audit. +The Conforma CLI version and the policy commit hash/tag resolved at validation time are recorded in each result row (`conforma_version` and `policy_version` respectively), enabling reproducibility and audit. #### Multi-tenancy Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. -### Structure of JSON returned from Conformat CLI validation request (from an example) +### Structure of JSON returned from Conforma CLI validation request (from an example) ```json { From 3b5149e7d07130d71182b363929fcf34bf0898eb Mon Sep 17 00:00:00 2001 From: arpcv Date: Thu, 5 Mar 2026 09:43:23 +0100 Subject: [PATCH 19/77] ec_validation_result is Singular --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 6847a6fb5..546552e63 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -315,7 +315,7 @@ sequenceDiagram - `configuration` (JSONB) - Branch, tag, auth credentials, etc. - `created_at`, `updated_at` (TIMESTAMP) -**`ec_validation_results`** - one row per validation execution +**`ec_validation_result`** - one row per validation execution - `id` (UUID, PK) - `validation_id` (VARCHAR, unique) - ID returned by the EC Wrapper, used for callback correlation From 1625541b32594f0e6085653a3f396d8cfb5297d4 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 11 Mar 2026 10:24:54 +0100 Subject: [PATCH 20/77] Adding more details --- .../00014-enterprise-contract-integration.md | 73 ++++++++++++------- 1 file changed, 48 insertions(+), 25 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 546552e63..a94b7cb25 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -34,7 +34,7 @@ An EC Wrapper (HTTP service) acts as a proxy between Trustify's EC service and C Each SBOM + policy pair has a validation state that follows this lifecycle: -- **Pending** — initial state, set when an SBOM is associated with a policy (e.g., a default policy assigned at SBOM upload time; upload itself is outside the scope of this ADR). Indicates no validation has been triggered yet for this SBOM against this policy. +- **Pending** — initial state, set when an SBOM is associated with a policy. Indicates no validation has been triggered yet for this SBOM against this policy. - **In Progress** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. - **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. - **Fail** — Conforma validation found policy violations; violation details are linked. @@ -74,10 +74,6 @@ Conforma is not available as WASM and would require major upstream changes. A Redis/RabbitMQ queue would improve retry handling and priority management; implement if the 429-based rejection approach proves insufficient under real load. -### Future API Migration - -When Conforma provides a REST API, the EC Wrapper can be replaced by pointing Trustify's adapter directly at the Conforma REST endpoint. A feature flag (`ec-api-mode`) allows gradual migration. No changes to the service layer, API endpoints, or UI are required. - ## The solution ### System Architecture @@ -108,10 +104,7 @@ C4Context ```mermaid C4Container title Enterprise Contract Integration - Container Diagram - Person(user, "Trustify User", "Software engineer or security analyst") - - Container_Boundary(trustify, "Trustify System") { Container(webui, "Web UI", "Rust/Actix", "Trustify GUI") Container(api, "API Gateway", "Actix-web", "REST API endpoints for SBOM
and compliance operations") @@ -130,24 +123,33 @@ C4Container System_Ext(policyRepo, "Policy Repository", "Git repository with EC policies") } + Container_Boundary(oidc, "OIDC") { + System_Ext(oidc, "OIDC", "OPenID Connect OAuth 2.0 + ", "") + } + Rel(user, webui, "Views compliance status", "HTTP API") Rel(user, api, "Views compliance status", "HTTP API") Rel(webui, api, "API calls", "JSON/HTTP API") Rel(api, ecModule, "Triggers validation", "Function call") - Rel(ecModule, ecWrapper, "POST /api/v1/validation {SBOM, policy_ref}
← returns validation_id", "HTTP API") - Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "HTTP callback") + Rel(ecModule, ecWrapper, "POST /validation", "HTTP API formData") + Rel(ecWrapper, api, "POST /validation/{id}/result", "HTTP API") Rel(ecWrapper, conforma, "ec validate input {SBOM} {policy}", "Spawned command") Rel(ecModule, postgres, "Saves validation
results", "SQL") Rel(ecModule, storage, "Stores EC reports", "Function call") Rel(storage, s3, "Persists reports", "S3 API") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") + Rel(ecWrapper, oidc, "Authenticate", "OAuth API") UpdateRelStyle(user, webui, $offsetX="-60", $offsetY="30") UpdateRelStyle(user, api, $offsetX="-60", $offsetY="-50") UpdateRelStyle(webui, api, $offsetX="-40", $offsetY="10") UpdateRelStyle(ecModule, ecWrapper, $offsetX="-50", $offsetY="-20") + UpdateRelStyle(ecWrapper, api, $offsetX="-60", $offsetY="-10") UpdateRelStyle(ecModule, postgres, $offsetX="-40", $offsetY="10") UpdateRelStyle(storage, s3, $offsetX="-40", $offsetY="10") + UpdateRelStyle(conforma, policyRepo, $offsetX="-40", $offsetY="100") + UpdateRelStyle(ecWrapper, oidc, $offsetX="30", $offsetY="40") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` @@ -180,7 +182,7 @@ C4Component Container_Boundary(dbms, "Database") { ContainerDb(postgres, "PostgreSQL", "Database", "Stores validation results and policy references") } - Container_Boundary(storage, "S3 System") { + Container_Boundary(storage, "S3 System") { System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") } @@ -188,7 +190,7 @@ C4Component Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT ec_policies", "SQL") - Rel(ecService, ecWrapper, "POST /api/v1/validation → returns validation_id", "HTTP") + Rel(ecService, ecWrapper, "POST /api/v1/validation → returns {id}", "HTTP") Rel(ecWrapper, conforma, "ec validate", "Process spawn") Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") Rel(ecService, resultParser, "parse_output()", "Function call") @@ -220,7 +222,7 @@ sequenceDiagram participant S3 as Object Storage participant Wrapper as EC Wrapper (HTTP) - User->>API: POST /api/v2/sbom/{sbom_id}/ec-validate {policy_id} + User->>API: POST /api/v2/sbom/{sbom_id}/ec-validate/{policy_id} API->>EP: dispatch request EP->>VS: validate_sbom(sbom_id, policy_id) @@ -318,36 +320,46 @@ sequenceDiagram **`ec_validation_result`** - one row per validation execution - `id` (UUID, PK) -- `validation_id` (VARCHAR, unique) - ID returned by the EC Wrapper, used for callback correlation - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → ec_policies) -- `status` (VARCHAR) - 'pending', 'in_progress', 'pass', 'fail', 'error' +- `status` (ENUM) - 'pending', 'in_progress', 'pass', 'fail', 'error' - `violations` (JSONB) - Structured violation data for querying - `summary` (JSONB) - Total checks, passed, failed, warnings - `report_url` (VARCHAR) - S3 URL to detailed report -- `executed_at` (TIMESTAMP) -- `execution_duration_ms` (INTEGER) +- `start_time` (TIMESTAMP) +- `end_time` (TIMESTAMP) - `conforma_version` (VARCHAR) - Conforma CLI version used (e.g., `v0.8.83`), for reproducibility - `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status -### API Endpoints +### Trustify API Endpoints ``` -POST /api/v2/sboms/{id}/ec-validate # Trigger validation +POST /api/v2/sboms/{id}/ec-validate # Trigger validation GET /api/v2/sboms/{id}/ec-report # Get latest validation result GET /api/v2/sboms/{id}/ec-report/history # Get validation history GET /api/v2/ec/report/{result_id} # Download detailed report from S3 POST /api/v2/ec/validation/{validation_id}/result # Callback: EC Wrapper posts Conforma result -POST /api/v2/ec/policies # Create policy reference (admin) -GET /api/v2/ec/policies # List policy references -GET /api/v2/ec/policies/{id} # Get policy reference -PUT /api/v2/ec/policies/{id} # Update policy reference (admin) -DELETE /api/v2/ec/policies/{id} # Delete policy reference (admin) +POST /api/v2/ec/policy # Create policy reference (admin) +GET /api/v2/ec/policy # List policy references +GET /api/v2/ec/policy/{id} # Get policy reference +PUT /api/v2/ec/policy/{id} # Update policy reference (admin) +DELETE /api/v2/ec/policy/{id} # Delete policy reference (admin) ``` -### Module Structure +## Conforma HTTP Wrapper API Endpoints + +### POST `/api/v1/validate` + +Validate the uploaded SBOM file against the provided rule URL. + +#### Response + +- 200 - if the validation request was accepted +- 401 - if the user was not authenticated + +### Trustify Module Structure ``` modules/ec/ @@ -369,6 +381,17 @@ modules/ec/ └── error.rs # Error types ``` +### HTTP Wrapper Module Structure + +``` +├── Cargo.toml +└── server + ├── lib.rs + ├── endpoints/ + │ └── mod.rs # REST endpoints + └── error.rs # Error types +``` + ### Technical Considerations #### Conforma CLI Execution (EC Wrapper) From 5fe077f67ddd918c287ca7c7cc6c2a0f638146cc Mon Sep 17 00:00:00 2001 From: arpcv Date: Thu, 12 Mar 2026 12:12:05 +0100 Subject: [PATCH 21/77] Use of a default Policy; Storage system and report_path --- docs/adrs/00014-enterprise-contract-integration.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index a94b7cb25..d0362bd58 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -34,7 +34,7 @@ An EC Wrapper (HTTP service) acts as a proxy between Trustify's EC service and C Each SBOM + policy pair has a validation state that follows this lifecycle: -- **Pending** — initial state, set when an SBOM is associated with a policy. Indicates no validation has been triggered yet for this SBOM against this policy. +- **Pending** — initial state, indicates no validation has been triggered yet for this SBOM against this policy. - **In Progress** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. - **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. - **Fail** — Conforma validation found policy violations; violation details are linked. @@ -45,10 +45,10 @@ The "In Progress" state serves as a concurrency guard: if a validation is alread What is stored where - PostgreSQL: validation status, structured violations (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, status, executed_at. -- S3/Minio: full raw Conforma JSON report, linked from the DB row via report_url. Keeps DB rows small while preserving audit completeness. +- Storage system: full raw Conforma JSON report, linked from the DB row via report_path. Keeps DB rows small while preserving audit completeness. - Not stored: the policy definitions themselves. ec_policies stores references (URLs, OCI refs) that Conforma fetches at runtime. -Storing full JSON in S3 rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB violations JSONB holds enough structure for filtering and dashboards without duplicating the full payload. +Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB violations JSONB holds enough structure for filtering and dashboards without duplicating the full payload. ## Consequences @@ -294,11 +294,11 @@ sequenceDiagram alt Pass VS->>DB: UPDATE ec_validation_results SET status='pass', violations=[] VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET report_url = ? + VS->>DB: UPDATE SET ureport_path = ? else Fail VS->>DB: UPDATE ec_validation_results SET status='fail', violations=json VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET report_url = ? + VS->>DB: UPDATE SET report_part = ? else Error VS->>DB: UPDATE ec_validation_results SET status='error', error_message=detail Note over VS,DB: Validation can be re-triggered (new request will create a new row with status='in_progress') @@ -325,7 +325,7 @@ sequenceDiagram - `status` (ENUM) - 'pending', 'in_progress', 'pass', 'fail', 'error' - `violations` (JSONB) - Structured violation data for querying - `summary` (JSONB) - Total checks, passed, failed, warnings -- `report_url` (VARCHAR) - S3 URL to detailed report +- `report_path` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) - `end_time` (TIMESTAMP) - `conforma_version` (VARCHAR) - Conforma CLI version used (e.g., `v0.8.83`), for reproducibility From 18e52a9345e288cf7a47aba4ad455b3d93352d99 Mon Sep 17 00:00:00 2001 From: arpcv Date: Thu, 12 Mar 2026 12:19:15 +0100 Subject: [PATCH 22/77] Explaining the Policy handling and the default one --- docs/adrs/00014-enterprise-contract-integration.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index d0362bd58..c1aff0e87 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -28,6 +28,8 @@ Users need the ability to: We will integrate Conforma into Trustify as a user triggered validation service by interacting with Conforma CLI. Validation is manually triggered — not automatic on SBOM upload. Validation on upload is deferred to a follow-up version. +Trustify stores information to identify (id, name, URL) of Policies. +A defaut Policy is defined at the application level which will be the Policy used for validation if a SBOM doesn't have a Policy attached to it. Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. An EC Wrapper (HTTP service) acts as a proxy between Trustify's EC service and Conforma CLI. From 1760a28c92102de4dc3913105d5e1d0f9cbac3d9 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 12 Mar 2026 17:49:35 +0100 Subject: [PATCH 23/77] Remove conforma_version --- docs/adrs/00014-enterprise-contract-integration.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index c1aff0e87..a1c4b92a1 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -330,7 +330,6 @@ sequenceDiagram - `report_path` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) - `end_time` (TIMESTAMP) -- `conforma_version` (VARCHAR) - Conforma CLI version used (e.g., `v0.8.83`), for reproducibility - `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status @@ -410,9 +409,9 @@ On the EC Wrapper side, concurrent Conforma processes are bounded by a semaphore #### Policy Management -ec_policies stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and must be encrypted at rest; they are never logged. +ec_policies stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. -The Conforma CLI version and the policy commit hash/tag resolved at validation time are recorded in each result row (`conforma_version` and `policy_version` respectively), enabling reproducibility and audit. +The policy commit hash/tag (`policy_version`) resolved at validation time are recorded in each result row, enabling reproducibility and audit. #### Multi-tenancy From e5fcc6b648c06f56d27d529b291ddcef758db489 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 12 Mar 2026 17:50:28 +0100 Subject: [PATCH 24/77] Replace with http wrapper --- docs/adrs/00014-enterprise-contract-integration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index a1c4b92a1..8a8e931c0 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -395,9 +395,9 @@ modules/ec/ ### Technical Considerations -#### Conforma CLI Execution (EC Wrapper) +#### Conforma CLI Execution (HTTP Wrapper) -The EC Wrapper invokes Conforma via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are written to a temp file and passed by path rather than piped via stdin, which avoids OOM issues. +The HTTP Wrapper invokes Conforma via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are written to a temp file and passed by path rather than piped via stdin, which avoids OOM issues. Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected failure, not an error), 2+ = execution error. It is important to distinguish 1 from 2+ in error handling — a policy violation is a valid result that should be surfaced to the user, not treated as a system failure. From 7fc848656a6b3226e54f32c8f0a96002ab3460c4 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 12 Mar 2026 18:22:23 +0100 Subject: [PATCH 25/77] HTTP Wrapper API --- docs/adrs/00014-enterprise-contract-integration.md | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 8a8e931c0..7ba45e535 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -351,14 +351,9 @@ DELETE /api/v2/ec/policy/{id} # Delete policy reference (admin) ## Conforma HTTP Wrapper API Endpoints -### POST `/api/v1/validate` - -Validate the uploaded SBOM file against the provided rule URL. - -#### Response - -- 200 - if the validation request was accepted -- 401 - if the user was not authenticated +``` +POST /api/v1/validate # Validate uploaded SBOM file against the provided Policy URL +``` ### Trustify Module Structure @@ -395,7 +390,7 @@ modules/ec/ ### Technical Considerations -#### Conforma CLI Execution (HTTP Wrapper) +#### Conforma CLI Execution (HTTPEC Wrapper) The HTTP Wrapper invokes Conforma via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are written to a temp file and passed by path rather than piped via stdin, which avoids OOM issues. From f59066dab13f34c628f18f29ad5f066df1b69f6d Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 17 Mar 2026 15:17:39 +0100 Subject: [PATCH 26/77] Two states --- .../00014-enterprise-contract-integration.md | 71 ++++++++++++------- 1 file changed, 44 insertions(+), 27 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 7ba45e535..3be4d16a4 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -29,26 +29,35 @@ We will integrate Conforma into Trustify as a user triggered validation service Validation is manually triggered — not automatic on SBOM upload. Validation on upload is deferred to a follow-up version. Trustify stores information to identify (id, name, URL) of Policies. -A defaut Policy is defined at the application level which will be the Policy used for validation if a SBOM doesn't have a Policy attached to it. +A default Policy is defined at the application level (global policy) which is used for validation when an SBOM does not have any Policy explicitly attached to it. +An SBOM can also carry references to one or more Policies. This one-to-many relationship between SBOM and Policy is modeled through a join table (`sbom_ec_policy`). Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. An EC Wrapper (HTTP service) acts as a proxy between Trustify's EC service and Conforma CLI. -Each SBOM + policy pair has a validation state that follows this lifecycle: +Each SBOM + policy pair has two validation states . + +The validation process state of the EC Wrapper follows this lifecycle: + +- **Queued** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. +- **In Progress** — the request has been submitted to EC Wrapper. +- **Completed** — the outcome of the request has been received back from EC Wrapper. +- **Failed** — an execution error occurred (CLI crash, policy fetch failure, timeout). The error is surfaced separately, and the validation can be re-triggered. + +The Policy validation outcome follows this lifecycle: - **Pending** — initial state, indicates no validation has been triggered yet for this SBOM against this policy. -- **In Progress** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. -- **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. - **Fail** — Conforma validation found policy violations; violation details are linked. -- **Error** — an execution error occurred (CLI crash, policy fetch failure, timeout). The error is surfaced separately, and the validation can be re-triggered. +- **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. +- **Error** — The Conforma validation has generated an error. -The "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. +The `ec_status` "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. What is stored where -- PostgreSQL: validation status, structured violations (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, status, executed_at. +- PostgreSQL: validation process state (`ec_status`), validation outcome (`status`), structured violations (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, ec_status, status, start_time. - Storage system: full raw Conforma JSON report, linked from the DB row via report_path. Keeps DB rows small while preserving audit completeness. -- Not stored: the policy definitions themselves. ec_policies stores references (URLs, OCI refs) that Conforma fetches at runtime. +- Not stored: the policy definitions themselves. ec_policy stores references (URLs, OCI refs) that Conforma fetches at runtime. Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB violations JSONB holds enough structure for filtering and dashboards without duplicating the full payload. @@ -191,13 +200,13 @@ C4Component Rel(api, ecEndpoints, "POST /sboms/{id}/ec-validate,\nGET /sboms/{id}/ec-report", "JSON/HTTPS") Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") - Rel(policyManager, postgres, "SELECT ec_policies", "SQL") + Rel(policyManager, postgres, "SELECT ec_policy", "SQL") Rel(ecService, ecWrapper, "POST /api/v1/validation → returns {id}", "HTTP") Rel(ecWrapper, conforma, "ec validate", "Process spawn") Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") Rel(ecService, resultParser, "parse_output()", "Function call") Rel(ecService, resultPersistence, "save_results()", "Function call") - Rel(resultPersistence, postgres, "INSERT ec_validation_results", "SQL") + Rel(resultPersistence, postgres, "INSERT ec_validation_result", "SQL") Rel(ecService, s3, "Store EC report", "S3 API") UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") @@ -229,7 +238,7 @@ sequenceDiagram EP->>VS: validate_sbom(sbom_id, policy_id) VS->>PM: get_policy_configuration(policy_id) - PM->>DB: SELECT * FROM ec_policies WHERE id = ? + PM->>DB: SELECT * FROM ec_policy WHERE id = ? alt Policy not found PM-->>VS: Error: PolicyNotFound VS-->>EP: 404 Not Found @@ -243,7 +252,7 @@ sequenceDiagram EP-->>API: 404 Not Found end - VS->>DB: SELECT * FROM ec_validation_results WHERE sbom_id = ? AND policy_id = ? AND status = 'in_progress' + VS->>DB: SELECT * FROM ec_validation_result WHERE sbom_id = ? AND policy_id = ? AND ec_status IN ('queued', 'in_progress') alt Validation already in progress VS-->>EP: 409 Conflict {existing job_id} EP-->>API: 409 Conflict @@ -256,7 +265,7 @@ sequenceDiagram VS->>Wrapper: POST /api/v1/validation {SBOM, policy_ref} Wrapper-->>VS: 202 Accepted {validation_id} - VS->>DB: INSERT ec_validation_results (status='in_progress', sbom_id, policy_id, validation_id, ...) + VS->>DB: INSERT ec_validation_result (ec_status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) VS-->>EP: 202 Accepted {validation_id} EP-->>API: 202 Accepted {validation_id} API-->>User: 202 Accepted {validation_id} @@ -294,22 +303,22 @@ sequenceDiagram EP->>VS: process_validation_result(validation_id, result) alt Pass - VS->>DB: UPDATE ec_validation_results SET status='pass', violations=[] + VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='pass', violations=[] VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET ureport_path = ? + VS->>DB: UPDATE SET report_path = ? else Fail - VS->>DB: UPDATE ec_validation_results SET status='fail', violations=json + VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='fail', violations=json VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET report_part = ? + VS->>DB: UPDATE SET report_path = ? else Error - VS->>DB: UPDATE ec_validation_results SET status='error', error_message=detail - Note over VS,DB: Validation can be re-triggered (new request will create a new row with status='in_progress') + VS->>DB: UPDATE ec_validation_result SET ec_status='failed', status='error', error_message=detail + Note over VS,DB: Validation can be re-triggered (new row with ec_status='in_progress', status='pending') end ``` ### The Data Model -**`ec_policies`** - Stores references to external policies, not the policies themselves +**`ec_policy`** - Stores references to external policies, not the policies themselves - `id` (UUID, PK) - `name` (VARCHAR, unique) - User-friendly name label @@ -319,12 +328,19 @@ sequenceDiagram - `configuration` (JSONB) - Branch, tag, auth credentials, etc. - `created_at`, `updated_at` (TIMESTAMP) +**`sbom_ec_policy`** - Join table linking SBOMs to their attached policies (one SBOM → many policies) + +- `sbom_id` (UUID, FK → sbom, PK) +- `policy_id` (UUID, FK → ec_policy, PK) +- `created_at` (TIMESTAMP) + **`ec_validation_result`** - one row per validation execution - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) -- `policy_id` (UUID, FK → ec_policies) -- `status` (ENUM) - 'pending', 'in_progress', 'pass', 'fail', 'error' +- `policy_id` (UUID, FK → ec_policy) +- `ec_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' +- `status` (ENUM) - 'pending', 'pass', 'fail', 'error' - `violations` (JSONB) - Structured violation data for querying - `summary` (JSONB) - Total checks, passed, failed, warnings - `report_path` (VARCHAR) - File system or S3 path to detailed report @@ -336,9 +352,10 @@ sequenceDiagram ### Trustify API Endpoints ``` -POST /api/v2/sboms/{id}/ec-validate # Trigger validation -GET /api/v2/sboms/{id}/ec-report # Get latest validation result -GET /api/v2/sboms/{id}/ec-report/history # Get validation history +POST /api/v2/sbom/{id}/ec-validate # Trigger validation +GET /api/v2/sbom/{id}/ec-report # Get latest validation result +GET /api/v2/sbom/{id}/ec-report/history # Get validation history + GET /api/v2/ec/report/{result_id} # Download detailed report from S3 POST /api/v2/ec/validation/{validation_id}/result # Callback: EC Wrapper posts Conforma result @@ -400,11 +417,11 @@ Temp files (SBOM, any cached policy material) are cleaned up in a finally-equiva #### Concurrency and Backpressure -On the EC Wrapper side, concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the EC Wrapper returns 429 Too Many Requests to Trustify, which propagates the status to the caller. This makes the capacity limit explicit to callers (e.g., CI pipelines can implement their own retry with backoff). On the Trustify side, the "In Progress" concurrency guard (409 Conflict) prevents duplicate validation runs for the same SBOM + policy pair. If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. +On the EC Wrapper side, concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the EC Wrapper returns 429 Too Many Requests to Trustify, which propagates the status to the caller. This makes the capacity limit explicit to callers (e.g., CI pipelines can implement their own retry with backoff). On the Trustify side, the `ec_status` "In Progress" concurrency guard (409 Conflict) prevents duplicate validation runs for the same SBOM + policy pair. If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. #### Policy Management -ec_policies stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. +ec_policy stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. The policy commit hash/tag (`policy_version`) resolved at validation time are recorded in each result row, enabling reproducibility and audit. From 5ff83b5b52b1540ab2a78aef8a2c1f1c359632b3 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 17 Mar 2026 15:52:56 +0100 Subject: [PATCH 27/77] Remove no used table --- .../00014-enterprise-contract-integration.md | 25 +++++++------------ 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 3be4d16a4..2443006e9 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -30,7 +30,6 @@ Validation is manually triggered — not automatic on SBOM upload. Validation on upload is deferred to a follow-up version. Trustify stores information to identify (id, name, URL) of Policies. A default Policy is defined at the application level (global policy) which is used for validation when an SBOM does not have any Policy explicitly attached to it. -An SBOM can also carry references to one or more Policies. This one-to-many relationship between SBOM and Policy is modeled through a join table (`sbom_ec_policy`). Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. An EC Wrapper (HTTP service) acts as a proxy between Trustify's EC service and Conforma CLI. @@ -328,12 +327,6 @@ sequenceDiagram - `configuration` (JSONB) - Branch, tag, auth credentials, etc. - `created_at`, `updated_at` (TIMESTAMP) -**`sbom_ec_policy`** - Join table linking SBOMs to their attached policies (one SBOM → many policies) - -- `sbom_id` (UUID, FK → sbom, PK) -- `policy_id` (UUID, FK → ec_policy, PK) -- `created_at` (TIMESTAMP) - **`ec_validation_result`** - one row per validation execution - `id` (UUID, PK) @@ -352,24 +345,24 @@ sequenceDiagram ### Trustify API Endpoints ``` -POST /api/v2/sbom/{id}/ec-validate # Trigger validation -GET /api/v2/sbom/{id}/ec-report # Get latest validation result -GET /api/v2/sbom/{id}/ec-report/history # Get validation history - -GET /api/v2/ec/report/{result_id} # Download detailed report from S3 -POST /api/v2/ec/validation/{validation_id}/result # Callback: EC Wrapper posts Conforma result - POST /api/v2/ec/policy # Create policy reference (admin) GET /api/v2/ec/policy # List policy references GET /api/v2/ec/policy/{id} # Get policy reference PUT /api/v2/ec/policy/{id} # Update policy reference (admin) DELETE /api/v2/ec/policy/{id} # Delete policy reference (admin) + +POST /api/v2/ec/validate?sbom_id={id}&policy_id={id} # Trigger validation +GET /api/v2/ec/report?sbom_id={id}&policy_id={id} # Get latest validation result +GET /api/v2/ec/report/history?sbom_id={id}&policy_id={id} # Get validation history +GET /api/v2/ec/report/{result_id} # Download detailed report from S3 + +POST /api/v2/ec/validation/{validation_id}/result # Callback: EC Wrapper posts Conforma result ``` -## Conforma HTTP Wrapper API Endpoints +## Conforma EC Wrapper API Endpoints ``` -POST /api/v1/validate # Validate uploaded SBOM file against the provided Policy URL +POST /api/v1/validate # Validate uploaded SBOM file against the provided Policy URL (multipart form) ``` ### Trustify Module Structure From 35db0a4cb3c4f4428aeffe36b6f9cbf5c2856115 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 17 Mar 2026 19:38:46 +0100 Subject: [PATCH 28/77] Add ec_policy.configuration model --- .../00014-enterprise-contract-integration.md | 70 +++++++++++++++++-- 1 file changed, 66 insertions(+), 4 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 2443006e9..f60f44ccb 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -322,11 +322,41 @@ sequenceDiagram - `id` (UUID, PK) - `name` (VARCHAR, unique) - User-friendly name label - `description` (TEXT) - What this policy enforces -- `policy_ref` (VARCHAR) - Git URL, OCI registry, or file path -- `policy_type` (VARCHAR) - 'git', 'oci', 'local' -- `configuration` (JSONB) - Branch, tag, auth credentials, etc. +- `policy_type` (VARCHAR) - 'Git URL' +- `configuration` (JSONB) - See model below - `created_at`, `updated_at` (TIMESTAMP) +**`ec_policy.configuration` JSONB model:** + +| Field | Type | Required | Description | +| ---------------------- | -------- | --------------- | -------------------------------------------------------------------------------- | +| `policy_ref` | string | yes | Policy source URL, e.g. `"git://github.com/org/policy-repo?ref=main"` | +| `auth` | object | no | Credentials for private repos; sensitive values encrypted via AES (never logged) | +| `auth.type` | string | yes (if `auth`) | `"token"`, `"ssh_key"`, or `"none"` | +| `auth.token_encrypted` | string | no | AES-encrypted bearer/PAT token, prefixed with encryption scheme | +| `policy_paths` | string[] | no | Sub-paths within the repo to evaluate (maps to Conforma `--policy` source paths) | +| `exclude` | string[] | no | Rule codes to skip during validation | +| `include` | string[] | no | If non-empty, only these rule codes are evaluated | +| `timeout_seconds` | integer | no | Per-policy override of the default 5-minute execution timeout | +| `extra_args` | string[] | no | Additional CLI flags forwarded verbatim to Conforma | + +Example of configuration field : + +```json +{ + "policy_ref": "git://github.com/org/policy-repo?ref=main", + "auth": { + "type": "token", + "token_encrypted": "AES256:" + }, + "policy_paths": ["policy/lib", "policy/release"], + "exclude": ["hello_world.minimal_packages"], + "include": [], + "timeout_seconds": 300, + "extra_args": ["--strict"] +} +``` + **`ec_validation_result`** - one row per validation execution - `id` (UUID, PK) @@ -334,7 +364,7 @@ sequenceDiagram - `policy_id` (UUID, FK → ec_policy) - `ec_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' - `status` (ENUM) - 'pending', 'pass', 'fail', 'error' -- `violations` (JSONB) - Structured violation data for querying +- `violations` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings - `report_path` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) @@ -342,6 +372,38 @@ sequenceDiagram - `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status +**`ec_validation_result.violations` JSONB model:** + +```json +[ + { + "severity": "violation", + "msg": "There are 2942 packages which is more than the permitted maximum of 510.", + "code": "hello_world.minimal_packages", + "title": "Check we don't have too many packages", + "description": "Just an example... To exclude this rule add \"hello_world.minimal_packages\" to the `exclude` section of the policy configuration.", + "solution": "You need to reduce the number of dependencies in this artifact." + }, + { + "severity": "warning", + "msg": "Deprecated license format detected.", + "code": "license.format_check", + "title": "License format validation", + "description": "Checks that license identifiers follow the SPDX specification.", + "solution": "Update license identifiers to valid SPDX expressions." + } +] +``` + +| Field | Type | Required | Description | +|---|---|---|---| +| `severity` | string | yes | `"violation"` or `"warning"` (derived from Conforma's `violations` vs `warnings` arrays) | +| `msg` | string | yes | Human-readable message describing the finding | +| `code` | string | yes | Rule identifier (e.g. `"hello_world.minimal_packages"`), useful for filtering and deduplication | +| `title` | string | yes | Short rule title | +| `description` | string | no | Longer explanation of what the rule checks | +| `solution` | string | no | Suggested remediation | + ### Trustify API Endpoints ``` From c6224f0d32baa9516443c3b5c0dbda6620f1a345 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 17 Mar 2026 19:55:15 +0100 Subject: [PATCH 29/77] Add ec_policy.results model --- .../00014-enterprise-contract-integration.md | 102 +++++++----------- 1 file changed, 36 insertions(+), 66 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index f60f44ccb..9aef3de9a 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -340,7 +340,7 @@ sequenceDiagram | `timeout_seconds` | integer | no | Per-policy override of the default 5-minute execution timeout | | `extra_args` | string[] | no | Additional CLI flags forwarded verbatim to Conforma | -Example of configuration field : +`ec_policy.configuration` example : ```json { @@ -364,7 +364,7 @@ Example of configuration field : - `policy_id` (UUID, FK → ec_policy) - `ec_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' - `status` (ENUM) - 'pending', 'pass', 'fail', 'error' -- `violations` (JSONB) - See model below +- `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings - `report_path` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) @@ -372,38 +372,54 @@ Example of configuration field : - `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status -**`ec_validation_result.violations` JSONB model:** +**`ec_validation_result.results` JSONB model:** + +| Field | Type | Required | Description | +| ---------------------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------------- | +| `severity` | string | yes | `"violation"`, `"warning"`, or `"success"` (derived from Conforma's `violations`, `warnings`, `successes` arrays) | +| `msg` | string | yes | Human-readable message describing the check outcome | +| `metadata` | object | yes | Rule metadata, preserved as-is from Conforma CLI output | +| `metadata.code` | string | yes | Rule identifier (e.g. `"hello_world.minimal_packages"`), useful for filtering and deduplication | +| `metadata.title` | string | yes | Short rule title | +| `metadata.description` | string | no | Longer explanation of what the rule checks | +| `metadata.solution` | string | no | Suggested remediation (typically absent for successes) | + +`ec_validation_result.results` example: ```json [ { "severity": "violation", "msg": "There are 2942 packages which is more than the permitted maximum of 510.", - "code": "hello_world.minimal_packages", - "title": "Check we don't have too many packages", - "description": "Just an example... To exclude this rule add \"hello_world.minimal_packages\" to the `exclude` section of the policy configuration.", - "solution": "You need to reduce the number of dependencies in this artifact." + "metadata": { + "code": "hello_world.minimal_packages", + "title": "Check we don't have too many packages", + "description": "Just an example... To exclude this rule add \"hello_world.minimal_packages\" to the `exclude` section of the policy configuration.", + "solution": "You need to reduce the number of dependencies in this artifact." + } }, { "severity": "warning", "msg": "Deprecated license format detected.", - "code": "license.format_check", - "title": "License format validation", - "description": "Checks that license identifiers follow the SPDX specification.", - "solution": "Update license identifiers to valid SPDX expressions." + "metadata": { + "code": "license.format_check", + "title": "License format validation", + "description": "Checks that license identifiers follow the SPDX specification.", + "solution": "Update license identifiers to valid SPDX expressions." + } + }, + { + "severity": "success", + "msg": "Pass", + "metadata": { + "code": "hello_world.valid_spdxid", + "title": "Check for valid SPDXID value", + "description": "Make sure that the SPDXID value found in the SBOM matches a list of allowed values." + } } ] ``` -| Field | Type | Required | Description | -|---|---|---|---| -| `severity` | string | yes | `"violation"` or `"warning"` (derived from Conforma's `violations` vs `warnings` arrays) | -| `msg` | string | yes | Human-readable message describing the finding | -| `code` | string | yes | Rule identifier (e.g. `"hello_world.minimal_packages"`), useful for filtering and deduplication | -| `title` | string | yes | Short rule title | -| `description` | string | no | Longer explanation of what the rule checks | -| `solution` | string | no | Suggested remediation | - ### Trustify API Endpoints ``` @@ -484,52 +500,6 @@ The policy commit hash/tag (`policy_version`) resolved at validation time are re Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. -### Structure of JSON returned from Conforma CLI validation request (from an example) - -```json -{ - "success": false, - "filepaths": [ - { - "filepath": "sboms/registry.redhat.io__rhtas__ec-rhel9__sha256__ea49a30eef5a2948b04540666a12048dd082625ce9f755acd3ece085c7d7937e.json", - "violations": [ - { - "msg": "There are 2942 packages which is more than the permitted maximum of 510.", - "metadata": { - "code": "hello_world.minimal_packages", - "description": "Just an example... To exclude this rule add \"hello_world.minimal_packages\" to the `exclude` section of the policy configuration.", - "solution": "You need to reduce the number of dependencies in this artifact.", - "title": "Check we don't have too many packages" - } - } - ], - "warnings": [], - "successes": [ - { - "msg": "Pass", - "metadata": { - "code": "hello_world.valid_spdxid", - "description": "Make sure that the SPDXID value found in the SBOM matches a list of allowed values.", - "title": "Check for valid SPDXID value" - } - } - ], - "success": false, - "success-count": 1 - } - ], - "policy": { - "sources": [ - { - "policy": ["github.com/conforma/policy//policy/lib", "./policy"] - } - ] - }, - "ec-version": "v0.8.83", - "effective-time": "2026-03-03T14:36:55.807826709Z" -} -``` - ### References - [Enterprise Contract (Conforma) GitHub](https://github.com/enterprise-contract/ec-cli) From d9cfae4c1754e9b67d69dc0cb88a3588b014e4b3 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 17 Mar 2026 20:02:57 +0100 Subject: [PATCH 30/77] Add ec_validation.summary model --- .../00014-enterprise-contract-integration.md | 47 ++++++++++++++----- 1 file changed, 35 insertions(+), 12 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 9aef3de9a..0e3d17d98 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -367,22 +367,19 @@ sequenceDiagram - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings - `report_path` (VARCHAR) - File system or S3 path to detailed report -- `start_time` (TIMESTAMP) -- `end_time` (TIMESTAMP) -- `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status **`ec_validation_result.results` JSONB model:** -| Field | Type | Required | Description | -| ---------------------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------------- | -| `severity` | string | yes | `"violation"`, `"warning"`, or `"success"` (derived from Conforma's `violations`, `warnings`, `successes` arrays) | -| `msg` | string | yes | Human-readable message describing the check outcome | -| `metadata` | object | yes | Rule metadata, preserved as-is from Conforma CLI output | -| `metadata.code` | string | yes | Rule identifier (e.g. `"hello_world.minimal_packages"`), useful for filtering and deduplication | -| `metadata.title` | string | yes | Short rule title | -| `metadata.description` | string | no | Longer explanation of what the rule checks | -| `metadata.solution` | string | no | Suggested remediation (typically absent for successes) | +| Field | Type | Required | Description | +| ---------------------- | ------ | -------- | ------------------------------------------------------- | +| `severity` | string | yes | `"violation"`, `"warning"`, or `"success"` | +| `msg` | string | yes | Human-readable message describing the check outcome | +| `metadata` | object | yes | Rule metadata, preserved as-is from Conforma CLI output | +| `metadata.code` | string | yes | Rule identifier for filtering and deduplication | +| `metadata.title` | string | yes | Short rule title | +| `metadata.description` | string | no | Detailed explanation of what the rule checks | +| `metadata.solution` | string | no | Suggested remediation (absent for successes) | `ec_validation_result.results` example: @@ -420,6 +417,32 @@ sequenceDiagram ] ``` +**`ec_validation_result.summary` JSONB model:** + +| Field | Type | Required | Description | +| ---------------- | ------- | -------- | ------------------------------------------------------------------------ | +| `success` | boolean | yes | Overall pass/fail outcome (mirrors Conforma's top-level `success` field) | +| `total` | integer | yes | Total number of checks evaluated | +| `violations` | integer | yes | Count of checks with violation severity | +| `warnings` | integer | yes | Count of checks with warning severity | +| `successes` | integer | yes | Count of checks that passed | +| `ec_version` | string | yes | Conforma version used (e.g. `"v0.8.83"`) | +| `effective_time` | string | yes | ISO 8601 timestamp of evaluation from Conforma | + +`ec_validation_result.summary` example: + +```json +{ + "success": false, + "total": 3, + "violations": 1, + "warnings": 1, + "successes": 1, + "ec_version": "v0.8.83", + "effective_time": "2026-03-03T14:36:55.807826709Z" +} +``` + ### Trustify API Endpoints ``` From 35590fd50d56da0a14f9fba38f41e8404ca618e8 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 17 Mar 2026 20:08:44 +0100 Subject: [PATCH 31/77] Fix inconsistencies --- .../00014-enterprise-contract-integration.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 0e3d17d98..d6f46746d 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -54,11 +54,11 @@ The `ec_status` "In Progress" state serves as a concurrency guard: if a validati What is stored where -- PostgreSQL: validation process state (`ec_status`), validation outcome (`status`), structured violations (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, ec_status, status, start_time. +- PostgreSQL: validation process state (`ec_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, ec_status, status, start_time. - Storage system: full raw Conforma JSON report, linked from the DB row via report_path. Keeps DB rows small while preserving audit completeness. - Not stored: the policy definitions themselves. ec_policy stores references (URLs, OCI refs) that Conforma fetches at runtime. -Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB violations JSONB holds enough structure for filtering and dashboards without duplicating the full payload. +Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB results JSONB holds enough structure for filtering and dashboards without duplicating the full payload. ## Consequences @@ -196,7 +196,7 @@ C4Component System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") } - Rel(api, ecEndpoints, "POST /sboms/{id}/ec-validate,\nGET /sboms/{id}/ec-report", "JSON/HTTPS") + Rel(api, ecEndpoints, "POST /ec/validate,\nGET /ec/report", "JSON/HTTPS") Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT ec_policy", "SQL") @@ -232,7 +232,7 @@ sequenceDiagram participant S3 as Object Storage participant Wrapper as EC Wrapper (HTTP) - User->>API: POST /api/v2/sbom/{sbom_id}/ec-validate/{policy_id} + User->>API: POST /api/v2/ec/validate (multipart form: sbom_id, policy_id) API->>EP: dispatch request EP->>VS: validate_sbom(sbom_id, policy_id) @@ -261,7 +261,7 @@ sequenceDiagram VS->>S3: retrieve_sbom_document(sbom_id) S3-->>VS: SBOM document (JSON/XML) - VS->>Wrapper: POST /api/v1/validation {SBOM, policy_ref} + VS->>Wrapper: POST /api/v1/validate {SBOM, policy_ref} Wrapper-->>VS: 202 Accepted {validation_id} VS->>DB: INSERT ec_validation_result (ec_status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) @@ -302,11 +302,11 @@ sequenceDiagram EP->>VS: process_validation_result(validation_id, result) alt Pass - VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='pass', violations=[] + VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='pass', results=[] VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_path = ? else Fail - VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='fail', violations=json + VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='fail', results=json VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_path = ? else Error @@ -367,6 +367,9 @@ sequenceDiagram - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings - `report_path` (VARCHAR) - File system or S3 path to detailed report +- `start_time` (TIMESTAMP) +- `end_time` (TIMESTAMP) +- `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status **`ec_validation_result.results` JSONB model:** @@ -452,7 +455,7 @@ GET /api/v2/ec/policy/{id} # Get policy reference PUT /api/v2/ec/policy/{id} # Update policy reference (admin) DELETE /api/v2/ec/policy/{id} # Delete policy reference (admin) -POST /api/v2/ec/validate?sbom_id={id}&policy_id={id} # Trigger validation +POST /api/v2/ec/validate # Trigger validation (multipart form: sbom_id, policy_id) GET /api/v2/ec/report?sbom_id={id}&policy_id={id} # Get latest validation result GET /api/v2/ec/report/history?sbom_id={id}&policy_id={id} # Get validation history GET /api/v2/ec/report/{result_id} # Download detailed report from S3 From d1a97b0fbfe764df72efb5d83c2ddc5632bbd219 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 17 Mar 2026 20:20:15 +0100 Subject: [PATCH 32/77] Flesh out Consequences of EC Wrapper --- .../00014-enterprise-contract-integration.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index d6f46746d..251679d76 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -62,7 +62,21 @@ Storing full JSON in storage system rather than only a summary was chosen explic ## Consequences -Using an EC Wrapper decouples the validation process into an external service. This better caters for large-scale deployments as EC validation has its own resource constraints. Meanwhile it adds infrastructure complexity as the EC Wrapper must be deployed and maintained alongside the Conforma CLI. +### Why EC runs externally + +EC validation can be be very resource-intensive (especially for large SBOMs with thousands of packages) and it should not compete with Trustify. +A dedicated EC Wrapper running alonside EC instance (Conforma CLI, etc) provides : + +- **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. +- **Independent scaling** — The EC Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. +- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure in `ec_status` and remains fully operational; the validation can be re-triggered. +- **Version independence** — The EC Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. + +The trade-off is added infrastructure complexity: the EC Wrapper must be deployed separatly with EC instance, monitored, and maintained as a separate component alongside the Conforma CLI binary. + +In Kubernetes or standalone machine deployments, the EC Wrapper pod has its own resource requests/limits, independent of the Trustify pod. + +### CLI spawning Within the EC Wrapper, Conforma is invoked via CLI spawning rather than a native API. This introduces an operational dependency (Conforma must be installed and version-pinned on every EC Wrapper instance) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma REST API exists yet. On the Trustify side, the EC service interacts with the EC Wrapper over HTTP and is built behind an adapter interface, so the implementation can be swapped for a direct Conforma REST client when one becomes available, without changes to the service layer or API. From 093c9a57f8e3de00343c6ec505dc1f9fd6f477e6 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 17 Mar 2026 20:23:44 +0100 Subject: [PATCH 33/77] Rephrase --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 251679d76..05a9b71fb 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -62,7 +62,7 @@ Storing full JSON in storage system rather than only a summary was chosen explic ## Consequences -### Why EC runs externally +### EC Wrapper runs externally EC validation can be be very resource-intensive (especially for large SBOMs with thousands of packages) and it should not compete with Trustify. A dedicated EC Wrapper running alonside EC instance (Conforma CLI, etc) provides : From 461e580b54fb624ae99ebf621917fe744b96180c Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Mar 2026 15:32:57 +0100 Subject: [PATCH 34/77] Create explicit futur work section --- docs/adrs/00014-enterprise-contract-integration.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 05a9b71fb..d7d130f5e 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -60,6 +60,12 @@ What is stored where Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB results JSONB holds enough structure for filtering and dashboards without duplicating the full payload. +### Futur work + +#### Multi-tenancy + +Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. + ## Consequences ### EC Wrapper runs externally @@ -536,10 +542,6 @@ ec_policy stores external references only. Conforma fetches the actual policy at The policy commit hash/tag (`policy_version`) resolved at validation time are recorded in each result row, enabling reproducibility and audit. -#### Multi-tenancy - -Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. - ### References - [Enterprise Contract (Conforma) GitHub](https://github.com/enterprise-contract/ec-cli) From 2c2828adc6955eb7aa5bf80de2783481688e148a Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Mar 2026 15:48:59 +0100 Subject: [PATCH 35/77] Rename ec_validation table --- .../00014-enterprise-contract-integration.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index d7d130f5e..01be0221d 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -225,7 +225,7 @@ C4Component Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") Rel(ecService, resultParser, "parse_output()", "Function call") Rel(ecService, resultPersistence, "save_results()", "Function call") - Rel(resultPersistence, postgres, "INSERT ec_validation_result", "SQL") + Rel(resultPersistence, postgres, "INSERT ec_validation", "SQL") Rel(ecService, s3, "Store EC report", "S3 API") UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") @@ -271,7 +271,7 @@ sequenceDiagram EP-->>API: 404 Not Found end - VS->>DB: SELECT * FROM ec_validation_result WHERE sbom_id = ? AND policy_id = ? AND ec_status IN ('queued', 'in_progress') + VS->>DB: SELECT * FROM ec_validation WHERE sbom_id = ? AND policy_id = ? AND ec_status IN ('queued', 'in_progress') alt Validation already in progress VS-->>EP: 409 Conflict {existing job_id} EP-->>API: 409 Conflict @@ -284,7 +284,7 @@ sequenceDiagram VS->>Wrapper: POST /api/v1/validate {SBOM, policy_ref} Wrapper-->>VS: 202 Accepted {validation_id} - VS->>DB: INSERT ec_validation_result (ec_status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) + VS->>DB: INSERT ec_validation (ec_status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) VS-->>EP: 202 Accepted {validation_id} EP-->>API: 202 Accepted {validation_id} API-->>User: 202 Accepted {validation_id} @@ -322,15 +322,15 @@ sequenceDiagram EP->>VS: process_validation_result(validation_id, result) alt Pass - VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='pass', results=[] + VS->>DB: UPDATE ec_validation SET ec_status='completed', status='pass', results=[] VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_path = ? else Fail - VS->>DB: UPDATE ec_validation_result SET ec_status='completed', status='fail', results=json + VS->>DB: UPDATE ec_validation SET ec_status='completed', status='fail', results=json VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_path = ? else Error - VS->>DB: UPDATE ec_validation_result SET ec_status='failed', status='error', error_message=detail + VS->>DB: UPDATE ec_validation SET ec_status='failed', status='error', error_message=detail Note over VS,DB: Validation can be re-triggered (new row with ec_status='in_progress', status='pending') end ``` @@ -377,7 +377,7 @@ sequenceDiagram } ``` -**`ec_validation_result`** - one row per validation execution +**`ec_validation`** - one row per validation execution - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) @@ -392,7 +392,7 @@ sequenceDiagram - `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status -**`ec_validation_result.results` JSONB model:** +**`ec_validation.results` JSONB model:** | Field | Type | Required | Description | | ---------------------- | ------ | -------- | ------------------------------------------------------- | @@ -404,7 +404,7 @@ sequenceDiagram | `metadata.description` | string | no | Detailed explanation of what the rule checks | | `metadata.solution` | string | no | Suggested remediation (absent for successes) | -`ec_validation_result.results` example: +`ec_validation.results` example: ```json [ @@ -440,7 +440,7 @@ sequenceDiagram ] ``` -**`ec_validation_result.summary` JSONB model:** +**`ec_validation.summary` JSONB model:** | Field | Type | Required | Description | | ---------------- | ------- | -------- | ------------------------------------------------------------------------ | @@ -452,7 +452,7 @@ sequenceDiagram | `ec_version` | string | yes | Conforma version used (e.g. `"v0.8.83"`) | | `effective_time` | string | yes | ISO 8601 timestamp of evaluation from Conforma | -`ec_validation_result.summary` example: +`ec_validation.summary` example: ```json { From 47b2ac590a7a767ca3fa3d76952b0525050660d4 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Mar 2026 15:52:48 +0100 Subject: [PATCH 36/77] ec_policy more generic with policy_type ENUM --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 01be0221d..3ee466475 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -342,7 +342,7 @@ sequenceDiagram - `id` (UUID, PK) - `name` (VARCHAR, unique) - User-friendly name label - `description` (TEXT) - What this policy enforces -- `policy_type` (VARCHAR) - 'Git URL' +- `policy_type` (ENUM) - 'Conforma' - `configuration` (JSONB) - See model below - `created_at`, `updated_at` (TIMESTAMP) From 86ddb6cc6bfe86a7cca4b1ac3aaaa30785b374e9 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Mar 2026 15:54:31 +0100 Subject: [PATCH 37/77] ec_policy.revision replace start/end datetime --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 3ee466475..9def16130 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -344,7 +344,7 @@ sequenceDiagram - `description` (TEXT) - What this policy enforces - `policy_type` (ENUM) - 'Conforma' - `configuration` (JSONB) - See model below -- `created_at`, `updated_at` (TIMESTAMP) +- `revision`(UUID) - Conditional UPDATE filtering both the primary key and the current revision **`ec_policy.configuration` JSONB model:** From 1d0f3eec70e078b7015c97c05525b5ba9cb43b5c Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Mar 2026 15:59:38 +0100 Subject: [PATCH 38/77] ec_validation: clarify status names --- docs/adrs/00014-enterprise-contract-integration.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 9def16130..0f2f706ff 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -382,10 +382,10 @@ sequenceDiagram - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → ec_policy) -- `ec_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' -- `status` (ENUM) - 'pending', 'pass', 'fail', 'error' +- `processing_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' +- `validation_status` (ENUM) - 'pending', 'pass', 'fail', 'error' - `results` (JSONB) - See model below -- `summary` (JSONB) - Total checks, passed, failed, warnings +- `summary` (JSONB) - Total checks, passed, failed, warnings, see model below - `report_path` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) - `end_time` (TIMESTAMP) @@ -450,7 +450,7 @@ sequenceDiagram | `warnings` | integer | yes | Count of checks with warning severity | | `successes` | integer | yes | Count of checks that passed | | `ec_version` | string | yes | Conforma version used (e.g. `"v0.8.83"`) | -| `effective_time` | string | yes | ISO 8601 timestamp of evaluation from Conforma | +| `effective_time` | string | yes | ISO 8601 timestamp of evaluation provided by Conforma | `ec_validation.summary` example: From 2c286ec70d4b04e85ac588ec965f75caa577d2fd Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Mar 2026 16:47:28 +0100 Subject: [PATCH 39/77] EC service: is now Policy verifier; Conforma used instead of EC name --- .../00014-enterprise-contract-integration.md | 126 +++++++++--------- 1 file changed, 63 insertions(+), 63 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 0f2f706ff..100922b7e 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -10,7 +10,7 @@ PROPOSED Trustify provides SBOM storage, analysis, and vulnerability tracking but lacks automated policy enforcement. Organizations need to validate SBOMs against security and compliance policies (licensing, vulnerabilities, provenance) without relying on manual, inconsistent review processes. -Enterprise Contract (Conforma) is an open-source policy enforcement tool actively maintained by Red Hat. It validates SBOMs against configurable policies and produces structured JSON output. Currently it provides only a CLI; a REST API is planned but with no committed timeline. +Conforma (former Enterprise Contract) is an open-source policy enforcement tool actively maintained by Red Hat. It validates SBOMs against configurable policies and produces structured JSON output. Currently it provides only a CLI; a REST API is planned but with no committed timeline. ### Requirements @@ -32,15 +32,15 @@ Trustify stores information to identify (id, name, URL) of Policies. A default Policy is defined at the application level (global policy) which is used for validation when an SBOM does not have any Policy explicitly attached to it. Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. -An EC Wrapper (HTTP service) acts as a proxy between Trustify's EC service and Conforma CLI. +A Conforma Wrapper (HTTP service) acts as a proxy between Trustify's Policy Verifier service and Conforma CLI. Each SBOM + policy pair has two validation states . -The validation process state of the EC Wrapper follows this lifecycle: +The validation process state of the Conforma Wrapper follows this lifecycle: - **Queued** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. -- **In Progress** — the request has been submitted to EC Wrapper. -- **Completed** — the outcome of the request has been received back from EC Wrapper. +- **In Progress** — the request has been submitted to Conforma Wrapper. +- **Completed** — the outcome of the request has been received back from Conforma Wrapper. - **Failed** — an execution error occurred (CLI crash, policy fetch failure, timeout). The error is surfaced separately, and the validation can be re-triggered. The Policy validation outcome follows this lifecycle: @@ -50,13 +50,13 @@ The Policy validation outcome follows this lifecycle: - **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. - **Error** — The Conforma validation has generated an error. -The `ec_status` "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. +The `processing_status` "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. What is stored where -- PostgreSQL: validation process state (`ec_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, ec_status, status, start_time. +- PostgreSQL: validation process state (`processing_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, processing_status, status, start_time. - Storage system: full raw Conforma JSON report, linked from the DB row via report_path. Keeps DB rows small while preserving audit completeness. -- Not stored: the policy definitions themselves. ec_policy stores references (URLs, OCI refs) that Conforma fetches at runtime. +- Not stored: the policy definitions themselves. policy stores references (URLs, OCI refs) that Conforma fetches at runtime. Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB results JSONB holds enough structure for filtering and dashboards without duplicating the full payload. @@ -68,23 +68,23 @@ Policy references are global (shared across all users) in this initial implement ## Consequences -### EC Wrapper runs externally +### The Conforma Wrapper runs externally EC validation can be be very resource-intensive (especially for large SBOMs with thousands of packages) and it should not compete with Trustify. -A dedicated EC Wrapper running alonside EC instance (Conforma CLI, etc) provides : +A dedicated Conforma Wrapper running alonside EC instance (Conforma CLI, etc) provides : - **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. -- **Independent scaling** — The EC Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. -- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure in `ec_status` and remains fully operational; the validation can be re-triggered. -- **Version independence** — The EC Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. +- **Independent scaling** — The Conforma Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. +- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure in `processing_status` and remains fully operational; the validation can be re-triggered. +- **Version independence** — The Conforma Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. -The trade-off is added infrastructure complexity: the EC Wrapper must be deployed separatly with EC instance, monitored, and maintained as a separate component alongside the Conforma CLI binary. +The trade-off is added infrastructure complexity: the Conforma Wrapper must be deployed separatly with EC instance, monitored, and maintained as a separate component alongside the Conforma CLI binary. -In Kubernetes or standalone machine deployments, the EC Wrapper pod has its own resource requests/limits, independent of the Trustify pod. +In Kubernetes or standalone machine deployments, the Conforma Wrapper pod has its own resource requests/limits, independent of the Trustify pod. ### CLI spawning -Within the EC Wrapper, Conforma is invoked via CLI spawning rather than a native API. This introduces an operational dependency (Conforma must be installed and version-pinned on every EC Wrapper instance) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma REST API exists yet. On the Trustify side, the EC service interacts with the EC Wrapper over HTTP and is built behind an adapter interface, so the implementation can be swapped for a direct Conforma REST client when one becomes available, without changes to the service layer or API. +Within the Conforma Wrapper, Conforma is invoked via CLI spawning rather than a native API. This introduces an operational dependency (Conforma must be installed and version-pinned on every Conforma Wrapper instance) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma REST API exists yet. On the Trustify side, the Policy Verifier service interacts with the Conforma Wrapper over HTTP and is built behind an adapter interface, so the implementation can be swapped for a direct Conforma REST client when one becomes available, without changes to the service layer or API. ### Alternatives Considered @@ -139,13 +139,13 @@ C4Container Container(webui, "Web UI", "Rust/Actix", "Trustify GUI") Container(api, "API Gateway", "Actix-web", "REST API endpoints for SBOM
and compliance operations") ContainerDb(postgres, "PostgreSQL", "DBMS", "Stores SBOM metadata, relationships,
and EC validation results") - Container(ecModule, "EC Validation Module", "Rust", "Orchestrates validation via
EC Wrapper and persists results") + Container(ecModule, "EC Validation Module", "Rust", "Orchestrates validation via
Conforma Wrapper and persists results") ContainerDb(s3, "Object Storage", "S3/Minio", "Stores SBOM documents and EC reports") Container(storage, "Storage Service", "Rust", "Manages document storage
(SBOMs, policy results)") } Container_Boundary(conformaSystem, "Conforma System") { - Container(ecWrapper, "EC Wrapper", "Rust/Actix", "HTTP Wrapper") + Container(ecWrapper, "Conforma Wrapper", "Rust/Actix", "HTTP Wrapper") System_Ext(conforma, "Conforma CLI", "External policy validation tool") } @@ -194,7 +194,7 @@ C4Component Container(api, "API Gateway", "Actix-web", "REST API for EC operations") Container_Boundary(ecModule, "EC Validation Module") { Container(ecEndpoints, "EC Endpoints", "Actix-web handlers", "REST endpoints for validation operations") - Component(ecService, "EC Service", "Business logic", "Orchestrates validation workflow") + Component(ecService, "Policy Verifier service", "Business logic", "Orchestrates validation workflow") Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output into structured data") Component(policyManager, "Policy Manager", "Business logic", "Manages EC policy references and
configuration") Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results") @@ -202,7 +202,7 @@ C4Component } Deployment_Node(external, "External System") { Deployment_Node(trustifyPod, "Trustify Pod") { - Component(ecWrapper, "EC Wrapper", "Actix-web handlers", "HTTP API") + Component(ecWrapper, "Conforma Wrapper", "Actix-web handlers", "HTTP API") } Deployment_Node(conformaPod, "Conforma Pod") { System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") @@ -219,13 +219,13 @@ C4Component Rel(api, ecEndpoints, "POST /ec/validate,\nGET /ec/report", "JSON/HTTPS") Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") Rel(ecService, policyManager, "get_policy_config()", "Function call") - Rel(policyManager, postgres, "SELECT ec_policy", "SQL") + Rel(policyManager, postgres, "SELECT policy", "SQL") Rel(ecService, ecWrapper, "POST /api/v1/validation → returns {id}", "HTTP") Rel(ecWrapper, conforma, "ec validate", "Process spawn") Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") Rel(ecService, resultParser, "parse_output()", "Function call") Rel(ecService, resultPersistence, "save_results()", "Function call") - Rel(resultPersistence, postgres, "INSERT ec_validation", "SQL") + Rel(resultPersistence, postgres, "INSERT policy_validation", "SQL") Rel(ecService, s3, "Store EC report", "S3 API") UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") @@ -246,18 +246,18 @@ sequenceDiagram actor User participant API as Trustify API participant EP as EC Endpoints - participant VS as EC Service + participant VS as Policy Verifier service participant PM as Policy Manager participant DB as PostgreSQL participant S3 as Object Storage - participant Wrapper as EC Wrapper (HTTP) + participant Wrapper as Conforma Wrapper (HTTP) User->>API: POST /api/v2/ec/validate (multipart form: sbom_id, policy_id) API->>EP: dispatch request EP->>VS: validate_sbom(sbom_id, policy_id) VS->>PM: get_policy_configuration(policy_id) - PM->>DB: SELECT * FROM ec_policy WHERE id = ? + PM->>DB: SELECT * FROM policy WHERE id = ? alt Policy not found PM-->>VS: Error: PolicyNotFound VS-->>EP: 404 Not Found @@ -271,7 +271,7 @@ sequenceDiagram EP-->>API: 404 Not Found end - VS->>DB: SELECT * FROM ec_validation WHERE sbom_id = ? AND policy_id = ? AND ec_status IN ('queued', 'in_progress') + VS->>DB: SELECT * FROM policy_validation WHERE sbom_id = ? AND policy_id = ? AND processing_status IN ('queued', 'in_progress') alt Validation already in progress VS-->>EP: 409 Conflict {existing job_id} EP-->>API: 409 Conflict @@ -284,7 +284,7 @@ sequenceDiagram VS->>Wrapper: POST /api/v1/validate {SBOM, policy_ref} Wrapper-->>VS: 202 Accepted {validation_id} - VS->>DB: INSERT ec_validation (ec_status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) + VS->>DB: INSERT policy_validation (processing_status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) VS-->>EP: 202 Accepted {validation_id} EP-->>API: 202 Accepted {validation_id} API-->>User: 202 Accepted {validation_id} @@ -297,10 +297,10 @@ sequenceDiagram autonumber participant API as Trustify API participant EP as EC Endpoints - participant VS as EC Service + participant VS as Policy Verifier service participant DB as PostgreSQL participant S3 as Object Storage - participant Wrapper as EC Wrapper (HTTP) + participant Wrapper as Conforma Wrapper (HTTP) participant Conf as Conforma CLI Note over Wrapper: Wrapper holds validation_id from the initial request @@ -322,22 +322,22 @@ sequenceDiagram EP->>VS: process_validation_result(validation_id, result) alt Pass - VS->>DB: UPDATE ec_validation SET ec_status='completed', status='pass', results=[] + VS->>DB: UPDATE policy_validation SET verification_status='completed', status='pass', results=[] VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_path = ? else Fail - VS->>DB: UPDATE ec_validation SET ec_status='completed', status='fail', results=json + VS->>DB: UPDATE policy_validation SET verification_status='completed', status='fail', results=json VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET report_path = ? else Error - VS->>DB: UPDATE ec_validation SET ec_status='failed', status='error', error_message=detail - Note over VS,DB: Validation can be re-triggered (new row with ec_status='in_progress', status='pending') + VS->>DB: UPDATE policy_validation SET verification_status='failed', status='error', error_message=detail + Note over VS,DB: Validation can be re-triggered (new row with processing_status='in_progress', status='pending') end ``` ### The Data Model -**`ec_policy`** - Stores references to external policies, not the policies themselves +**`policy`** - Stores references to external policies, not the policies themselves - `id` (UUID, PK) - `name` (VARCHAR, unique) - User-friendly name label @@ -346,7 +346,7 @@ sequenceDiagram - `configuration` (JSONB) - See model below - `revision`(UUID) - Conditional UPDATE filtering both the primary key and the current revision -**`ec_policy.configuration` JSONB model:** +**`policy.configuration` JSONB model:** | Field | Type | Required | Description | | ---------------------- | -------- | --------------- | -------------------------------------------------------------------------------- | @@ -360,7 +360,7 @@ sequenceDiagram | `timeout_seconds` | integer | no | Per-policy override of the default 5-minute execution timeout | | `extra_args` | string[] | no | Additional CLI flags forwarded verbatim to Conforma | -`ec_policy.configuration` example : +`policy.configuration` example : ```json { @@ -377,13 +377,13 @@ sequenceDiagram } ``` -**`ec_validation`** - one row per validation execution +**`policy_validation`** - one row per validation execution - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) -- `policy_id` (UUID, FK → ec_policy) +- `policy_id` (UUID, FK → policy) - `processing_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' -- `validation_status` (ENUM) - 'pending', 'pass', 'fail', 'error' +- `verification_status` (ENUM) - 'pending', 'pass', 'fail', 'error' - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings, see model below - `report_path` (VARCHAR) - File system or S3 path to detailed report @@ -392,7 +392,7 @@ sequenceDiagram - `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status -**`ec_validation.results` JSONB model:** +**`policy_validation.results` JSONB model:** | Field | Type | Required | Description | | ---------------------- | ------ | -------- | ------------------------------------------------------- | @@ -404,7 +404,7 @@ sequenceDiagram | `metadata.description` | string | no | Detailed explanation of what the rule checks | | `metadata.solution` | string | no | Suggested remediation (absent for successes) | -`ec_validation.results` example: +`policy_validation.results` example: ```json [ @@ -440,7 +440,7 @@ sequenceDiagram ] ``` -**`ec_validation.summary` JSONB model:** +**`policy_validation.summary` JSONB model:** | Field | Type | Required | Description | | ---------------- | ------- | -------- | ------------------------------------------------------------------------ | @@ -452,7 +452,7 @@ sequenceDiagram | `ec_version` | string | yes | Conforma version used (e.g. `"v0.8.83"`) | | `effective_time` | string | yes | ISO 8601 timestamp of evaluation provided by Conforma | -`ec_validation.summary` example: +`policy_validation.summary` example: ```json { @@ -469,21 +469,21 @@ sequenceDiagram ### Trustify API Endpoints ``` -POST /api/v2/ec/policy # Create policy reference (admin) -GET /api/v2/ec/policy # List policy references -GET /api/v2/ec/policy/{id} # Get policy reference -PUT /api/v2/ec/policy/{id} # Update policy reference (admin) -DELETE /api/v2/ec/policy/{id} # Delete policy reference (admin) - -POST /api/v2/ec/validate # Trigger validation (multipart form: sbom_id, policy_id) -GET /api/v2/ec/report?sbom_id={id}&policy_id={id} # Get latest validation result -GET /api/v2/ec/report/history?sbom_id={id}&policy_id={id} # Get validation history -GET /api/v2/ec/report/{result_id} # Download detailed report from S3 - -POST /api/v2/ec/validation/{validation_id}/result # Callback: EC Wrapper posts Conforma result +POST /api/v2/policy # Create policy reference (admin) +GET /api/v2/policy # List policy references +GET /api/v2/policy/{id} # Get policy reference +PUT /api/v2/policy/{id} # Update policy reference (admin) +DELETE /api/v2/policy/{id} # Delete policy reference (admin) + +POST /api/v2/policy/validate # Trigger validation (multipart form: sbom_id, policy_id) +GET /api/v2/policy/report?sbom_id={id}&policy_id={id} # Get latest validation result +GET /api/v2/policy/report/history?sbom_id={id}&policy_id={id} # Get validation history +GET /api/v2/policy/report/{result_id} # Download detailed report from S3 + +POST /api/v2/ec/validation/{validation_id}/result # Callback: Conforma Wrapper posts Conforma result ``` -## Conforma EC Wrapper API Endpoints +## Conforma Wrapper API Endpoints ``` POST /api/v1/validate # Validate uploaded SBOM file against the provided Policy URL (multipart form) @@ -492,7 +492,7 @@ POST /api/v1/validate # Validate uploaded SBOM file agains ### Trustify Module Structure ``` -modules/ec/ +modules/policy/ ├── Cargo.toml └── src/ ├── lib.rs @@ -506,12 +506,12 @@ modules/ec/ │ ├── mod.rs │ ├── ec_service.rs # Main orchestration │ ├── policy_manager.rs # Policy configuration - │ ├── executor.rs # EC Wrapper HTTP client (adapter) + │ ├── executor.rs # Conforma Wrapper HTTP client (adapter) │ └── result_parser.rs # Output parsing └── error.rs # Error types ``` -### HTTP Wrapper Module Structure +### Conforma Wrapper Module Structure ``` ├── Cargo.toml @@ -524,9 +524,9 @@ modules/ec/ ### Technical Considerations -#### Conforma CLI Execution (HTTPEC Wrapper) +#### Conforma CLI Execution -The HTTP Wrapper invokes Conforma via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are written to a temp file and passed by path rather than piped via stdin, which avoids OOM issues. +The Conforma Wrapper invokes Conforma CLI via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are written to a temp file and passed by path rather than piped via stdin, which avoids OOM issues. Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected failure, not an error), 2+ = execution error. It is important to distinguish 1 from 2+ in error handling — a policy violation is a valid result that should be surfaced to the user, not treated as a system failure. @@ -534,11 +534,11 @@ Temp files (SBOM, any cached policy material) are cleaned up in a finally-equiva #### Concurrency and Backpressure -On the EC Wrapper side, concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the EC Wrapper returns 429 Too Many Requests to Trustify, which propagates the status to the caller. This makes the capacity limit explicit to callers (e.g., CI pipelines can implement their own retry with backoff). On the Trustify side, the `ec_status` "In Progress" concurrency guard (409 Conflict) prevents duplicate validation runs for the same SBOM + policy pair. If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. +On the Conforma Wrapper side, concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the Conforma Wrapper returns 429 Too Many Requests to Trustify, which propagates the status to the caller. This makes the capacity limit explicit to callers (e.g., CI pipelines can implement their own retry with backoff). On the Trustify side, the `processing_status` "In Progress" concurrency guard (409 Conflict) prevents duplicate validation runs for the same SBOM + policy pair. If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. #### Policy Management -ec_policy stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. +policy stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. The policy commit hash/tag (`policy_version`) resolved at validation time are recorded in each result row, enabling reproducibility and audit. From 0dd10808fc4116ebd2b7ce183ce12e5fc2041853 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 18 Mar 2026 16:56:20 +0100 Subject: [PATCH 40/77] Diagrams: Replaced EC --- .../00014-enterprise-contract-integration.md | 82 +++++++++---------- 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 100922b7e..077c9d130 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -129,7 +129,7 @@ C4Context UpdateLayoutConfig($c4ShapeInRow="2", $c4BoundaryInRow="1") ``` -### Container Diagram - EC Validation Module +### Container Diagram - Policy Validation Module ```mermaid C4Container @@ -139,13 +139,13 @@ C4Container Container(webui, "Web UI", "Rust/Actix", "Trustify GUI") Container(api, "API Gateway", "Actix-web", "REST API endpoints for SBOM
and compliance operations") ContainerDb(postgres, "PostgreSQL", "DBMS", "Stores SBOM metadata, relationships,
and EC validation results") - Container(ecModule, "EC Validation Module", "Rust", "Orchestrates validation via
Conforma Wrapper and persists results") + Container(policyValidationModule, "Policy Validation Module", "Rust", "Orchestrates validation via
Conforma Wrapper and persists results") ContainerDb(s3, "Object Storage", "S3/Minio", "Stores SBOM documents and EC reports") Container(storage, "Storage Service", "Rust", "Manages document storage
(SBOMs, policy results)") } Container_Boundary(conformaSystem, "Conforma System") { - Container(ecWrapper, "Conforma Wrapper", "Rust/Actix", "HTTP Wrapper") + Container(conformaWrapper, "Conforma Wrapper", "Rust/Actix", "HTTP Wrapper") System_Ext(conforma, "Conforma CLI", "External policy validation tool") } @@ -161,25 +161,25 @@ C4Container Rel(user, webui, "Views compliance status", "HTTP API") Rel(user, api, "Views compliance status", "HTTP API") Rel(webui, api, "API calls", "JSON/HTTP API") - Rel(api, ecModule, "Triggers validation", "Function call") - Rel(ecModule, ecWrapper, "POST /validation", "HTTP API formData") - Rel(ecWrapper, api, "POST /validation/{id}/result", "HTTP API") - Rel(ecWrapper, conforma, "ec validate input {SBOM} {policy}", "Spawned command") - Rel(ecModule, postgres, "Saves validation
results", "SQL") - Rel(ecModule, storage, "Stores EC reports", "Function call") + Rel(api, policyValidationModule, "Triggers validation", "Function call") + Rel(policyValidationModule, conformaWrapper, "POST /validation", "HTTP API formData") + Rel(conformaWrapper, api, "POST /validation/{id}/result", "HTTP API") + Rel(conformaWrapper, conforma, "ec validate input {SBOM} {policy}", "Spawned command") + Rel(policyValidationModule, postgres, "Saves validation
results", "SQL") + Rel(policyValidationModule, storage, "Stores EC reports", "Function call") Rel(storage, s3, "Persists reports", "S3 API") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") - Rel(ecWrapper, oidc, "Authenticate", "OAuth API") + Rel(conformaWrapper, oidc, "Authenticate", "OAuth API") UpdateRelStyle(user, webui, $offsetX="-60", $offsetY="30") UpdateRelStyle(user, api, $offsetX="-60", $offsetY="-50") UpdateRelStyle(webui, api, $offsetX="-40", $offsetY="10") - UpdateRelStyle(ecModule, ecWrapper, $offsetX="-50", $offsetY="-20") - UpdateRelStyle(ecWrapper, api, $offsetX="-60", $offsetY="-10") - UpdateRelStyle(ecModule, postgres, $offsetX="-40", $offsetY="10") + UpdateRelStyle(policyValidationModule, conformaWrapper, $offsetX="-50", $offsetY="-20") + UpdateRelStyle(conformaWrapper, api, $offsetX="-60", $offsetY="-10") + UpdateRelStyle(policyValidationModule, postgres, $offsetX="-40", $offsetY="10") UpdateRelStyle(storage, s3, $offsetX="-40", $offsetY="10") UpdateRelStyle(conforma, policyRepo, $offsetX="-40", $offsetY="100") - UpdateRelStyle(ecWrapper, oidc, $offsetX="30", $offsetY="40") + UpdateRelStyle(conformaWrapper, oidc, $offsetX="30", $offsetY="40") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` @@ -188,13 +188,13 @@ C4Container ```mermaid C4Component - title EC Validation Module - Component Diagram + title Policy Validation Module - Component Diagram Deployment_Node(trustifySystem, "Trustify System") { Container(api, "API Gateway", "Actix-web", "REST API for EC operations") - Container_Boundary(ecModule, "EC Validation Module") { - Container(ecEndpoints, "EC Endpoints", "Actix-web handlers", "REST endpoints for validation operations") - Component(ecService, "Policy Verifier service", "Business logic", "Orchestrates validation workflow") + Container_Boundary(policyValidationModule, "Policy Validation Module") { + Container(policyEndpoints, "Policy Endpoints", "Actix-web handlers", "REST endpoints for validation operations") + Component(policyVeriferService, "Policy Verifier service", "Business logic", "Orchestrates validation workflow") Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output into structured data") Component(policyManager, "Policy Manager", "Business logic", "Manages EC policy references and
configuration") Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results") @@ -202,7 +202,7 @@ C4Component } Deployment_Node(external, "External System") { Deployment_Node(trustifyPod, "Trustify Pod") { - Component(ecWrapper, "Conforma Wrapper", "Actix-web handlers", "HTTP API") + Component(conformaWrapper, "Conforma Wrapper", "Actix-web handlers", "HTTP API") } Deployment_Node(conformaPod, "Conforma Pod") { System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") @@ -216,24 +216,24 @@ C4Component System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") } - Rel(api, ecEndpoints, "POST /ec/validate,\nGET /ec/report", "JSON/HTTPS") - Rel(ecEndpoints, ecService, "validate_sbom() / get_ec_report()", "Function call") - Rel(ecService, policyManager, "get_policy_config()", "Function call") + Rel(api, policyEndpoints, "POST /ec/validate,\nGET /ec/report", "JSON/HTTPS") + Rel(policyEndpoints, policyVeriferService, "validate_sbom() / get_ec_report()", "Function call") + Rel(policyVeriferService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT policy", "SQL") - Rel(ecService, ecWrapper, "POST /api/v1/validation → returns {id}", "HTTP") - Rel(ecWrapper, conforma, "ec validate", "Process spawn") - Rel(ecWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") - Rel(ecService, resultParser, "parse_output()", "Function call") - Rel(ecService, resultPersistence, "save_results()", "Function call") + Rel(policyVeriferService, conformaWrapper, "POST /api/v1/validation → returns {id}", "HTTP") + Rel(conformaWrapper, conforma, "ec validate", "Process spawn") + Rel(conformaWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") + Rel(policyVeriferService, resultParser, "parse_output()", "Function call") + Rel(policyVeriferService, resultPersistence, "save_results()", "Function call") Rel(resultPersistence, postgres, "INSERT policy_validation", "SQL") - Rel(ecService, s3, "Store EC report", "S3 API") + Rel(policyVeriferService, s3, "Store EC report", "S3 API") - UpdateRelStyle(api, ecEndpoints, $offsetX="-50", $offsetY="-50") - UpdateRelStyle(ecEndpoints, ecService, $offsetX="-60", $offsetY="+40") - UpdateRelStyle(ecService, ecWrapper, $offsetX="-20", $offsetY="10") - UpdateRelStyle(ecWrapper, api, $offsetX="20", $offsetY="-40") - UpdateRelStyle(ecService, resultParser, $offsetX="-60", $offsetY="+0") - UpdateRelStyle(ecService, resultPersistence, $offsetX="-60", $offsetY="+80") + UpdateRelStyle(api, policyEndpoints, $offsetX="-50", $offsetY="-50") + UpdateRelStyle(policyEndpoints, policyVeriferService, $offsetX="-60", $offsetY="+40") + UpdateRelStyle(policyVeriferService, conformaWrapper, $offsetX="-20", $offsetY="10") + UpdateRelStyle(conformaWrapper, api, $offsetX="20", $offsetY="-40") + UpdateRelStyle(policyVeriferService, resultParser, $offsetX="-60", $offsetY="+0") + UpdateRelStyle(policyVeriferService, resultPersistence, $offsetX="-60", $offsetY="+80") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` @@ -245,7 +245,7 @@ sequenceDiagram autonumber actor User participant API as Trustify API - participant EP as EC Endpoints + participant EP as Policy Endpoints participant VS as Policy Verifier service participant PM as Policy Manager participant DB as PostgreSQL @@ -296,7 +296,7 @@ sequenceDiagram sequenceDiagram autonumber participant API as Trustify API - participant EP as EC Endpoints + participant EP as Policy Endpoints participant VS as Policy Verifier service participant DB as PostgreSQL participant S3 as Object Storage @@ -475,12 +475,12 @@ GET /api/v2/policy/{id} # Get policy reference PUT /api/v2/policy/{id} # Update policy reference (admin) DELETE /api/v2/policy/{id} # Delete policy reference (admin) -POST /api/v2/policy/validate # Trigger validation (multipart form: sbom_id, policy_id) -GET /api/v2/policy/report?sbom_id={id}&policy_id={id} # Get latest validation result -GET /api/v2/policy/report/history?sbom_id={id}&policy_id={id} # Get validation history -GET /api/v2/policy/report/{result_id} # Download detailed report from S3 +POST /api/v2/policy/validate # Trigger validation (multipart form: sbom_id, policy_id) +GET /api/v2/policy/report?sbom_id={id}&policy_id={id} # Get latest validation result +GET /api/v2/policy/report/history?sbom_id={id}&policy_id={id} # Get validation history +GET /api/v2/policy/report/{result_id} # Download detailed report from S3 -POST /api/v2/ec/validation/{validation_id}/result # Callback: Conforma Wrapper posts Conforma result +POST /api/v2/policy/validation/{validation_id}/result # Callback: Conforma Wrapper posts Conforma result ``` ## Conforma Wrapper API Endpoints From 66bca4aa77341013fe01bedd11c223e33e608845 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 24 Mar 2026 12:07:50 +0100 Subject: [PATCH 41/77] Use JSON parameter --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 077c9d130..efc5a5d00 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -475,7 +475,7 @@ GET /api/v2/policy/{id} # Get policy reference PUT /api/v2/policy/{id} # Update policy reference (admin) DELETE /api/v2/policy/{id} # Delete policy reference (admin) -POST /api/v2/policy/validate # Trigger validation (multipart form: sbom_id, policy_id) +POST /api/v2/policy/validate?sbom_id={id}&policy_id={id} # Trigger validation GET /api/v2/policy/report?sbom_id={id}&policy_id={id} # Get latest validation result GET /api/v2/policy/report/history?sbom_id={id}&policy_id={id} # Get validation history GET /api/v2/policy/report/{result_id} # Download detailed report from S3 From ecc8d7b79c7728d407669d89998b1b429a486f3f Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 24 Mar 2026 12:08:14 +0100 Subject: [PATCH 42/77] Adr must be approved --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index efc5a5d00..e986c59c8 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -4,7 +4,7 @@ Date: 2026-02-03 ## Status -PROPOSED +APPROVED ## Context From 4bcfa51199ad35534b70b5d941331b5b5beb5348 Mon Sep 17 00:00:00 2001 From: arpcv Date: Tue, 24 Mar 2026 14:35:34 +0100 Subject: [PATCH 43/77] Actionable feedback is future work --- docs/adrs/00014-enterprise-contract-integration.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index e986c59c8..f49610ace 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -66,6 +66,10 @@ Storing full JSON in storage system rather than only a summary was chosen explic Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. +#### Receive actionable feedback on policy violations + +This is out of scope of this ADR. + ## Consequences ### The Conforma Wrapper runs externally From 109049ec16df7642e1da762825b9f70359173603 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 24 Mar 2026 14:39:11 +0100 Subject: [PATCH 44/77] Future work --- docs/adrs/00014-enterprise-contract-integration.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index f49610ace..ecf61b4e7 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -27,7 +27,6 @@ Users need the ability to: We will integrate Conforma into Trustify as a user triggered validation service by interacting with Conforma CLI. Validation is manually triggered — not automatic on SBOM upload. -Validation on upload is deferred to a follow-up version. Trustify stores information to identify (id, name, URL) of Policies. A default Policy is defined at the application level (global policy) which is used for validation when an SBOM does not have any Policy explicitly attached to it. @@ -62,6 +61,8 @@ Storing full JSON in storage system rather than only a summary was chosen explic ### Futur work +#### Validation on SBOM upload + #### Multi-tenancy Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. From 899d2d24f2e1dcf5f766a19933ffc128a2c86577 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 24 Mar 2026 15:08:18 +0100 Subject: [PATCH 45/77] Use source_document.id --- docs/adrs/00014-enterprise-contract-integration.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index ecf61b4e7..eb0d7fa6a 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -54,7 +54,7 @@ The `processing_status` "In Progress" state serves as a concurrency guard: if a What is stored where - PostgreSQL: validation process state (`processing_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, processing_status, status, start_time. -- Storage system: full raw Conforma JSON report, linked from the DB row via report_path. Keeps DB rows small while preserving audit completeness. +- Storage system: full raw Conforma JSON report, linked from the DB row via `source_document.id`. Keeps DB rows small while preserving audit completeness. - Not stored: the policy definitions themselves. policy stores references (URLs, OCI refs) that Conforma fetches at runtime. Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB results JSONB holds enough structure for filtering and dashboards without duplicating the full payload. @@ -329,11 +329,11 @@ sequenceDiagram alt Pass VS->>DB: UPDATE policy_validation SET verification_status='completed', status='pass', results=[] VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET report_path = ? + VS->>DB: UPDATE SET source_document.id = ? else Fail VS->>DB: UPDATE policy_validation SET verification_status='completed', status='fail', results=json VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET report_path = ? + VS->>DB: UPDATE SET source_document.id = ? else Error VS->>DB: UPDATE policy_validation SET verification_status='failed', status='error', error_message=detail Note over VS,DB: Validation can be re-triggered (new row with processing_status='in_progress', status='pending') @@ -391,7 +391,7 @@ sequenceDiagram - `verification_status` (ENUM) - 'pending', 'pass', 'fail', 'error' - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings, see model below -- `report_path` (VARCHAR) - File system or S3 path to detailed report +- `source_document_id` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) - `end_time` (TIMESTAMP) - `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time From b2590df213c268f685b23409e0a07aff6bf93e9f Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 24 Mar 2026 17:15:15 +0100 Subject: [PATCH 46/77] Move future section at bottom --- .../00014-enterprise-contract-integration.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index eb0d7fa6a..2b89dbc42 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -59,18 +59,6 @@ What is stored where Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB results JSONB holds enough structure for filtering and dashboards without duplicating the full payload. -### Futur work - -#### Validation on SBOM upload - -#### Multi-tenancy - -Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. - -#### Receive actionable feedback on policy violations - -This is out of scope of this ADR. - ## Consequences ### The Conforma Wrapper runs externally @@ -547,6 +535,18 @@ policy stores external references only. Conforma fetches the actual policy at va The policy commit hash/tag (`policy_version`) resolved at validation time are recorded in each result row, enabling reproducibility and audit. +### Futur work + +#### Validation on SBOM upload + +#### Multi-tenancy + +Policy references are global (shared across all users) in this initial implementation. Per-organization policy namespacing is out of scope here and should be addressed in a dedicated multi-tenancy ADR when Trustify adds org-level isolation more broadly. + +#### Receive actionable feedback on policy violations + +This is out of scope of this ADR. + ### References - [Enterprise Contract (Conforma) GitHub](https://github.com/enterprise-contract/ec-cli) From 81db590ed9a48a41dc8b163269002d9be105aa9b Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 24 Mar 2026 17:34:51 +0100 Subject: [PATCH 47/77] Use Conforma name instead of EC --- .../00014-enterprise-contract-integration.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 2b89dbc42..ef3a809c7 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -435,15 +435,15 @@ sequenceDiagram **`policy_validation.summary` JSONB model:** -| Field | Type | Required | Description | -| ---------------- | ------- | -------- | ------------------------------------------------------------------------ | -| `success` | boolean | yes | Overall pass/fail outcome (mirrors Conforma's top-level `success` field) | -| `total` | integer | yes | Total number of checks evaluated | -| `violations` | integer | yes | Count of checks with violation severity | -| `warnings` | integer | yes | Count of checks with warning severity | -| `successes` | integer | yes | Count of checks that passed | -| `ec_version` | string | yes | Conforma version used (e.g. `"v0.8.83"`) | -| `effective_time` | string | yes | ISO 8601 timestamp of evaluation provided by Conforma | +| Field | Type | Required | Description | +| ------------------ | ------- | -------- | ------------------------------------------------------------------------ | +| `success` | boolean | yes | Overall pass/fail outcome (mirrors Conforma's top-level `success` field) | +| `total` | integer | yes | Total number of checks evaluated | +| `violations` | integer | yes | Count of checks with violation severity | +| `warnings` | integer | yes | Count of checks with warning severity | +| `successes` | integer | yes | Count of checks that passed | +| `conforma_version` | string | yes | Conforma version used (e.g. `"v0.8.83"`) | +| `effective_time` | string | yes | ISO 8601 timestamp of evaluation provided by Conforma | `policy_validation.summary` example: @@ -454,7 +454,7 @@ sequenceDiagram "violations": 1, "warnings": 1, "successes": 1, - "ec_version": "v0.8.83", + "conforma_version": "v0.8.83", "effective_time": "2026-03-03T14:36:55.807826709Z" } ``` From e7322594b7e7eb44a63393b541f58b736d5569c9 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 25 Mar 2026 10:13:54 +0100 Subject: [PATCH 48/77] Move summary into table --- docs/adrs/00014-enterprise-contract-integration.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index ef3a809c7..c1ea989b1 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -378,7 +378,13 @@ sequenceDiagram - `processing_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' - `verification_status` (ENUM) - 'pending', 'pass', 'fail', 'error' - `results` (JSONB) - See model below -- `summary` (JSONB) - Total checks, passed, failed, warnings, see model below +- `success` (BOOLEAN) - Overall pass/fail outcome (mirrors Conforma's top-level `success` field) +- `total` (SMALLINT) - Total number of checks evaluated +- `violations` (SMALLINT) - Count of checks with violation severity +- `warnings` (SMALLINT) - Count of checks with warning severity +- `successes` (SMALLINT) - Count of checks that passed +- `conforma_version`(VARCHAR) - Conforma version used (e.g. `"v0.8.83"`) +- `effective_time` (VARCHAR) - ISO 8601 timestamp of evaluation provided by Conforma - `source_document_id` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) - `end_time` (TIMESTAMP) From b3078b6b8af0b5b41b31298bc51e6da17ffeab3c Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 26 Mar 2026 15:31:07 +0100 Subject: [PATCH 49/77] Policy managemt better phrasing; Wrapper is included in trustd --- .../00014-enterprise-contract-integration.md | 22 +++++++++---------- 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index c1ea989b1..cf49787f6 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -508,17 +508,13 @@ modules/policy/ │ ├── executor.rs # Conforma Wrapper HTTP client (adapter) │ └── result_parser.rs # Output parsing └── error.rs # Error types -``` - -### Conforma Wrapper Module Structure - -``` -├── Cargo.toml -└── server - ├── lib.rs - ├── endpoints/ - │ └── mod.rs # REST endpoints - └── error.rs # Error types +└── conforma_wrapper + ├── build.rs + ├── Cargo.toml + └── src/ + ├── endpoints/ + │ └── mod.rs # REST endpoints + └── lib.rs ``` ### Technical Considerations @@ -537,7 +533,9 @@ On the Conforma Wrapper side, concurrent Conforma processes are bounded by a sem #### Policy Management -policy stores external references only. Conforma fetches the actual policy at validation time, which means Trustify does not cache policy content by default. The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. +`policy` stores external references only as the policy is fetched by Conforma at validation time, therefore Trustify does not cache policy content. + +The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. The policy commit hash/tag (`policy_version`) resolved at validation time are recorded in each result row, enabling reproducibility and audit. From 00401fc45e9cedb12a6862834ed79110b3d07abb Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 26 Mar 2026 15:39:44 +0100 Subject: [PATCH 50/77] Renamed executor.rs to client/conforma.rs --- docs/adrs/00014-enterprise-contract-integration.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index cf49787f6..71a101f31 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -494,6 +494,7 @@ POST /api/v1/validate # Validate uploaded SBOM file agains modules/policy/ ├── Cargo.toml └── src/ + ├── error.rs # Error types ├── lib.rs ├── endpoints/ │ └── mod.rs # REST endpoints @@ -505,9 +506,9 @@ modules/policy/ │ ├── mod.rs │ ├── ec_service.rs # Main orchestration │ ├── policy_manager.rs # Policy configuration - │ ├── executor.rs # Conforma Wrapper HTTP client (adapter) │ └── result_parser.rs # Output parsing - └── error.rs # Error types + └── client/ + └── conforma.rs # Conforma client adapter └── conforma_wrapper ├── build.rs ├── Cargo.toml From 0c38166f0eb066adeb6a005f10579ee30fe9b681 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 26 Mar 2026 15:41:32 +0100 Subject: [PATCH 51/77] Fix tree --- .../00014-enterprise-contract-integration.md | 32 +++++++++---------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 71a101f31..adba9f183 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -493,22 +493,22 @@ POST /api/v1/validate # Validate uploaded SBOM file agains ``` modules/policy/ ├── Cargo.toml -└── src/ - ├── error.rs # Error types - ├── lib.rs - ├── endpoints/ - │ └── mod.rs # REST endpoints - ├── model/ - │ ├── mod.rs - │ ├── policy.rs # Policy API models - │ └── validation.rs # Validation result models - ├── service/ - │ ├── mod.rs - │ ├── ec_service.rs # Main orchestration - │ ├── policy_manager.rs # Policy configuration - │ └── result_parser.rs # Output parsing - └── client/ - └── conforma.rs # Conforma client adapter +├── src/ +│ ├── error.rs # Error types +│ ├── lib.rs +│ ├── endpoints/ +│ │ └── mod.rs # REST endpoints +│ ├── model/ +│ │ ├── mod.rs +│ │ ├── policy.rs # Policy API models +│ │ └── validation.rs # Validation result models +│ ├── service/ +│ │ ├── mod.rs +│ │ ├── ec_service.rs # Main orchestration +│ │ ├── policy_manager.rs # Policy configuration +│ │ └── result_parser.rs # Output parsing +│ └── client/ +│ └── conforma.rs # Conforma client adapter └── conforma_wrapper ├── build.rs ├── Cargo.toml From 4c7e3e653b261668122bb2dbaf924fb221c079a3 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 26 Mar 2026 15:42:56 +0100 Subject: [PATCH 52/77] File structure --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index adba9f183..a37edc99d 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -488,7 +488,7 @@ POST /api/v2/policy/validation/{validation_id}/result # Callback: Con POST /api/v1/validate # Validate uploaded SBOM file against the provided Policy URL (multipart form) ``` -### Trustify Module Structure +### Trustify File Structure ``` modules/policy/ From 2d9f2002d83d5c3dce54488fdc09081de56984bf Mon Sep 17 00:00:00 2001 From: arpcv Date: Mon, 30 Mar 2026 07:48:29 +0200 Subject: [PATCH 53/77] Expanded API Endpoints --- .../00014-enterprise-contract-integration.md | 304 +++++++++++++++++- 1 file changed, 289 insertions(+), 15 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index a37edc99d..e14eeb924 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -465,22 +465,268 @@ sequenceDiagram } ``` -### Trustify API Endpoints +### POST `/api/v2/policy` -``` -POST /api/v2/policy # Create policy reference (admin) -GET /api/v2/policy # List policy references -GET /api/v2/policy/{id} # Get policy reference -PUT /api/v2/policy/{id} # Update policy reference (admin) -DELETE /api/v2/policy/{id} # Delete policy reference (admin) - -POST /api/v2/policy/validate?sbom_id={id}&policy_id={id} # Trigger validation -GET /api/v2/policy/report?sbom_id={id}&policy_id={id} # Get latest validation result -GET /api/v2/policy/report/history?sbom_id={id}&policy_id={id} # Get validation history -GET /api/v2/policy/report/{result_id} # Download detailed report from S3 - -POST /api/v2/policy/validation/{validation_id}/result # Callback: Conforma Wrapper posts Conforma result -``` +Create a new policy reference. + +#### Request + +| part | name | type | description | +| ---- | ---- | --------------- | ----------- | +| body | - | `PolicyRequest` | | + +#### Response + +- 201 - the policy was created + + ```yaml + id: # ID of the created policy + ``` + + And: + + ``` + Location: /api/v2/policy/ + ``` + +- 400 - if the request could not be understood +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized +- 409 - if a policy with the same name already exists + +### GET `/api/v2/policy` + +List policy references, optionally filtered. + +By default, the entries will be sorted by name ascending. + +#### Request + +| part | name | type | description | +| ----- | -------- | ---------- | ------------------------------------------------------- | +| query | `q` | "q" string | "q style" query string | +| query | `limit` | u64 | Maximum number of items to return | +| query | `offset` | u64 | Initial items to skip before actually returning results | + +The following `q` parameters are supported: + +- `name`: Filters policies by their name. + +#### Response + +- 200 - if the user is allowed to read policies + + ```rust + #[derive(Serialize, Deserialize)] + struct PaginatedPolicy { + total: u64, + items: Vec, + } + ``` + +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized + +### GET `/api/v2/policy/{id}` + +Get a single policy reference by ID. + +#### Request + +| part | name | type | description | +| ---- | ---- | -------- | ----------------------- | +| path | `id` | `String` | ID of the policy to get | + +#### Response + +- 200 - if the policy was found + + | part | name | type | description | + | ------- | ------ | -------- | ---------------------------------- | + | body | - | `Policy` | The policy information | + | headers | `ETag` | string | Value which indicates the revision | + +- 401 - if the user was not authenticated +- 404 - if the policy was not found or the user doesn't have permission to read this policy + +### PUT `/api/v2/policy/{id}` + +Update an existing policy reference. + +#### Request + +| part | name | type | description | +| ------ | --------- | ---------------- | ------------------------------ | +| path | `id` | `String` | ID of the policy to update | +| header | `IfMatch` | `Option` | ETag value, revision to update | +| body | - | `PolicyRequest` | The new content | + +#### Response + +- 204 - the policy was updated +- 400 - if the request could not be understood +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized +- 404 - if the policy was not found +- 409 - if a policy with the same name already exists +- 412 - if the `IfMatch` header was present, but its value didn't match the stored revision + +### DELETE `/api/v2/policy/{id}` + +Delete an existing policy reference. + +Deleting a policy will fail if there are validation results referencing it. + +#### Request + +| part | name | type | description | +| ------ | --------- | ---------------- | ------------------------------ | +| path | `id` | `String` | ID of the policy to delete | +| header | `IfMatch` | `Option` | ETag value, revision to delete | + +#### Response + +- 204 - if the policy was successfully deleted +- 204 - if the policy was already deleted +- 400 - if the request could not be understood +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized +- 409 - if the policy has associated validation results +- 412 - if the `IfMatch` header was present, but its value didn't match the stored revision + +### POST `/api/v2/policy/validate` + +Trigger a policy validation for a given SBOM and policy pair. The validation is performed asynchronously by the Conforma Wrapper; a `validation_id` is returned immediately. + +If a validation is already in progress for the same SBOM + policy pair, the request is rejected with 409 Conflict. + +#### Request + +| part | name | type | description | +| ----- | ----------- | -------- | ----------------------------------------------------------------- | +| query | `sbom_id` | `String` | ID of the SBOM to validate | +| query | `policy_id` | `String` | ID of the policy to validate against (omit to use default policy) | + +#### Response + +- 202 - the validation was accepted and queued + + ```rust + #[derive(Serialize, Deserialize)] + struct ValidationAccepted { + validation_id: Uuid, + } + ``` + +- 400 - if the request could not be understood +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized +- 404 - if the SBOM or policy was not found +- 409 - if a validation is already in progress for this SBOM + policy pair +- 429 - if the Conforma Wrapper has reached its concurrency limit + +### GET `/api/v2/policy/report` + +Get the latest validation result for a given SBOM and policy pair. + +#### Request + +| part | name | type | description | +| ----- | ----------- | -------- | --------------------------------------------- | +| query | `sbom_id` | `String` | ID of the SBOM | +| query | `policy_id` | `String` | ID of the policy (omit to use default policy) | + +#### Response + +- 200 - if a validation result exists + + | part | name | type | description | + | ---- | ---- | ------------------ | ---------------------------- | + | body | - | `PolicyValidation` | The latest validation result | + +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized +- 404 - if the SBOM or policy was not found, or no validation has been performed yet + +### GET `/api/v2/policy/report/history` + +Get the validation history for a given SBOM and policy pair, ordered by `start_time` descending. + +#### Request + +| part | name | type | description | +| ----- | ----------- | ---------- | ------------------------------------------------------- | +| query | `sbom_id` | `String` | ID of the SBOM | +| query | `policy_id` | `String` | ID of the policy (omit to use default policy) | +| query | `q` | "q" string | "q style" query string | +| query | `limit` | u64 | Maximum number of items to return | +| query | `offset` | u64 | Initial items to skip before actually returning results | + +The following `q` parameters are supported: + +- `processing_status`: Filters by processing status (`queued`, `in_progress`, `completed`, `failed`). +- `verification_status`: Filters by verification status (`pending`, `pass`, `fail`, `error`). + +#### Response + +- 200 - if the SBOM and policy exist + + ```rust + #[derive(Serialize, Deserialize)] + struct PaginatedPolicyValidation { + total: u64, + items: Vec, + } + ``` + +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized +- 404 - if the SBOM or policy was not found + +### GET `/api/v2/policy/report/{result_id}` + +Download the full raw Conforma JSON report from storage. + +#### Request + +| part | name | type | description | +| ---- | ----------- | -------- | ------------------------------------ | +| path | `result_id` | `String` | ID of the validation result to fetch | + +#### Response + +- 200 - if the report was found + + | part | name | type | description | + | ------- | -------------- | -------- | ----------------------------- | + | body | - | raw JSON | The full Conforma JSON report | + | headers | `Content-Type` | string | `application/json` | + +- 401 - if the user was not authenticated +- 403 - if the user was authenticated but not authorized +- 404 - if the validation result or report was not found + +### POST `/api/v2/policy/validation/{validation_id}/result` + +Callback endpoint used by the Conforma Wrapper to post the validation result back to Trustify after Conforma CLI execution completes. + +This endpoint is not intended for end-user use. + +#### Request + +| part | name | type | description | +| ---- | --------------- | -------- | --------------------------------------------------- | +| path | `validation_id` | `String` | ID of the validation (returned in the 202 response) | +| body | - | raw JSON | The raw Conforma CLI JSON output | + +#### Response + +- 204 - the result was accepted and persisted +- 400 - if the request could not be understood or the JSON is malformed +- 401 - if the caller was not authenticated +- 403 - if the caller was not authorized +- 404 - if the validation ID was not found +- 409 - if the validation already has a result (duplicate callback) ## Conforma Wrapper API Endpoints @@ -488,6 +734,34 @@ POST /api/v2/policy/validation/{validation_id}/result # Callback: Con POST /api/v1/validate # Validate uploaded SBOM file against the provided Policy URL (multipart form) ``` +### POST `/api/v1/validate` + +Accept an SBOM document and policy reference, spawn a Conforma CLI validation, and asynchronously post the result back to the Trustify callback endpoint. + +#### Request + +| part | name | type | description | +| --------- | -------------- | -------- | -------------------------------------------------------------------------- | +| multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | +| multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | +| multipart | `callback_url` | `String` | Trustify callback URL (`/api/v2/policy/validation/{validation_id}/result`) | +| multipart | `extra_args` | `String` | Additional CLI flags forwarded to Conforma (optional, JSON-encoded array) | + +#### Response + +- 202 - the validation was accepted and will be processed asynchronously + + ```rust + #[derive(Serialize, Deserialize)] + struct WrapperValidationAccepted { + validation_id: Uuid, + } + ``` + +- 400 - if the request could not be understood or required fields are missing +- 401 - if the caller was not authenticated +- 429 - if the concurrency semaphore is exhausted (too many concurrent validations) + ### Trustify File Structure ``` From 7bbcae09efdf9169664e2a1cfc689aee9ae332ab Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 07:51:46 +0200 Subject: [PATCH 54/77] Detailed data model --- .../00014-enterprise-contract-integration.md | 106 ++++++++++++++++++ 1 file changed, 106 insertions(+) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index e14eeb924..0a22eff01 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -465,6 +465,112 @@ sequenceDiagram } ``` +#### Data Model Implementation + +```rust +/// The policy reference information +#[derive(Serialize, Deserialize)] +struct Policy { + id: Uuid, + name: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + description: Option, + policy_type: String, + configuration: PolicyConfiguration, +} +``` + +```rust +/// Policy information that can be mutated +#[derive(Serialize, Deserialize)] +struct PolicyRequest { + name: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + description: Option, + policy_type: String, + configuration: PolicyConfiguration, +} +``` + +```rust +/// Policy configuration (stored as JSONB) +#[derive(Serialize, Deserialize)] +struct PolicyConfiguration { + policy_ref: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + auth: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + policy_paths: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + exclude: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + include: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + timeout_seconds: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + extra_args: Vec, +} +``` + +```rust +/// Validation result summary returned by the API +#[derive(Serialize, Deserialize)] +struct PolicyValidation { + id: Uuid, + sbom_id: Uuid, + policy_id: Uuid, + processing_status: String, + verification_status: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + success: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + total: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + violations: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + warnings: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + successes: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + conforma_version: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + effective_time: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + results: Option>, + #[serde(default, skip_serializing_if = "Option::is_none")] + source_document_id: Option, + start_time: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + end_time: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + policy_version: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + error_message: Option, +} +``` + +```rust +/// A single check result within a validation +#[derive(Serialize, Deserialize)] +struct PolicyValidationResult { + severity: String, + msg: String, + metadata: PolicyValidationResultMetadata, +} +``` + +```rust +#[derive(Serialize, Deserialize)] +struct PolicyValidationResultMetadata { + code: String, + title: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + description: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + solution: Option, +} +``` + ### POST `/api/v2/policy` Create a new policy reference. From b55b98669f400073475e6836f2a42f8c80d330a6 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 08:01:23 +0200 Subject: [PATCH 55/77] All SBOMs are transferred via file --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 0a22eff01..e4a011663 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -902,7 +902,7 @@ modules/policy/ #### Conforma CLI Execution -The Conforma Wrapper invokes Conforma CLI via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); large SBOMs are written to a temp file and passed by path rather than piped via stdin, which avoids OOM issues. +The Conforma Wrapper invokes Conforma CLI via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); SBOMs are written to a temp file and passed by path in order to avoid OOM issues as SBOM can be very large file they shouldn't not be transfered via STDIN stream. Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected failure, not an error), 2+ = execution error. It is important to distinguish 1 from 2+ in error handling — a policy violation is a valid result that should be surfaced to the user, not treated as a system failure. From 0a92de31e0c7d1afc99c08eb0b1ace1b26f4205e Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 08:03:12 +0200 Subject: [PATCH 56/77] Remove obsolete text --- docs/adrs/00014-enterprise-contract-integration.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index e4a011663..b60c90d4d 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -906,8 +906,6 @@ The Conforma Wrapper invokes Conforma CLI via process spawning (e.g., `tokio::pr Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected failure, not an error), 2+ = execution error. It is important to distinguish 1 from 2+ in error handling — a policy violation is a valid result that should be surfaced to the user, not treated as a system failure. -Temp files (SBOM, any cached policy material) are cleaned up in a finally-equivalent block regardless of execution outcome, including on timeout. - #### Concurrency and Backpressure On the Conforma Wrapper side, concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the Conforma Wrapper returns 429 Too Many Requests to Trustify, which propagates the status to the caller. This makes the capacity limit explicit to callers (e.g., CI pipelines can implement their own retry with backoff). On the Trustify side, the `processing_status` "In Progress" concurrency guard (409 Conflict) prevents duplicate validation runs for the same SBOM + policy pair. If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. From 245243f8fd5b47ee6c2d34d7eb50378fa247ea5a Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 08:43:35 +0200 Subject: [PATCH 57/77] Failed state is terminal --- .../00014-enterprise-contract-integration.md | 23 +++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index b60c90d4d..4387ef21a 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -40,7 +40,17 @@ The validation process state of the Conforma Wrapper follows this lifecycle: - **Queued** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. - **In Progress** — the request has been submitted to Conforma Wrapper. - **Completed** — the outcome of the request has been received back from Conforma Wrapper. -- **Failed** — an execution error occurred (CLI crash, policy fetch failure, timeout). The error is surfaced separately, and the validation can be re-triggered. +- **Failed** — an execution error occurred (CLI crash, policy fetch failure, timeout). This is a terminal state; the error is surfaced to the user, who may manually queue a new validation run. + +```mermaid +stateDiagram-v2 + [*] --> Queued : User triggers validation + Queued --> InProgress : Request submitted to\nConforma Wrapper + InProgress --> Completed : Outcome received + InProgress --> Failed : Execution error\n(crash, timeout, fetch failure) + Completed --> [*] + Failed --> [*] +``` The Policy validation outcome follows this lifecycle: @@ -68,7 +78,7 @@ A dedicated Conforma Wrapper running alonside EC instance (Conforma CLI, etc) pr - **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. - **Independent scaling** — The Conforma Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. -- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure in `processing_status` and remains fully operational; the validation can be re-triggered. +- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure as a terminal `processing_status` and remains fully operational; the user may manually queue a new validation. - **Version independence** — The Conforma Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. The trade-off is added infrastructure complexity: the Conforma Wrapper must be deployed separatly with EC instance, monitored, and maintained as a separate component alongside the Conforma CLI binary. @@ -324,7 +334,7 @@ sequenceDiagram VS->>DB: UPDATE SET source_document.id = ? else Error VS->>DB: UPDATE policy_validation SET verification_status='failed', status='error', error_message=detail - Note over VS,DB: Validation can be re-triggered (new row with processing_status='in_progress', status='pending') + Note over VS,DB: Failed is terminal. User may manually queue a new validation (new row). end ``` @@ -908,7 +918,12 @@ Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected fai #### Concurrency and Backpressure -On the Conforma Wrapper side, concurrent Conforma processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the Conforma Wrapper returns 429 Too Many Requests to Trustify, which propagates the status to the caller. This makes the capacity limit explicit to callers (e.g., CI pipelines can implement their own retry with backoff). On the Trustify side, the `processing_status` "In Progress" concurrency guard (409 Conflict) prevents duplicate validation runs for the same SBOM + policy pair. If demand grows to warrant it, a proper queue (Redis/RabbitMQ) is the deferred alternative considered below. +Concurrency is controlled at two levels: + +- **Trustify (duplicate prevention)** — Before forwarding a request to the Conforma Wrapper, the Policy Verifier service checks whether a validation is already queued or in progress for the same SBOM + policy pair. If one exists, the request is rejected with 409 Conflict, preventing duplicate work. +- **Conforma Wrapper (resource protection)** — Concurrent Conforma CLI processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the Wrapper returns 429 Too Many Requests, which Trustify propagates to the caller. This makes the capacity limit explicit so that callers (e.g., CI pipelines) can implement their own retry with backoff. + +If demand grows beyond what the semaphore-based approach can handle, a proper queue (Redis/RabbitMQ) is a deferred alternative (see _Batch Processing Queue_ in Alternatives Considered above). #### Policy Management From 30e3eb4266529b3980e9e20a5d9c4fe4134a5760 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 09:34:55 +0200 Subject: [PATCH 58/77] Clarify conforma_version and policy_version --- docs/adrs/00014-enterprise-contract-integration.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 4387ef21a..973230258 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -353,7 +353,7 @@ sequenceDiagram | Field | Type | Required | Description | | ---------------------- | -------- | --------------- | -------------------------------------------------------------------------------- | -| `policy_ref` | string | yes | Policy source URL, e.g. `"git://github.com/org/policy-repo?ref=main"` | +| `policy_ref` | string | yes | Policy source URL, e.g. `"git://[URL]?ref=[BRANCH OR TAG]"` | | `auth` | object | no | Credentials for private repos; sensitive values encrypted via AES (never logged) | | `auth.type` | string | yes (if `auth`) | `"token"`, `"ssh_key"`, or `"none"` | | `auth.token_encrypted` | string | no | AES-encrypted bearer/PAT token, prefixed with encryption scheme | @@ -398,7 +398,6 @@ sequenceDiagram - `source_document_id` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) - `end_time` (TIMESTAMP) -- `policy_version` (VARCHAR) - Policy commit hash or tag resolved at validation time - `error_message` (TEXT) - Populated only on error status **`policy_validation.results` JSONB model:** @@ -458,7 +457,7 @@ sequenceDiagram | `violations` | integer | yes | Count of checks with violation severity | | `warnings` | integer | yes | Count of checks with warning severity | | `successes` | integer | yes | Count of checks that passed | -| `conforma_version` | string | yes | Conforma version used (e.g. `"v0.8.83"`) | +| `conforma_version` | string | yes | Version of Conforma CLI (e.g. `"v0.8.83"`) | | `effective_time` | string | yes | ISO 8601 timestamp of evaluation provided by Conforma | `policy_validation.summary` example: @@ -927,11 +926,14 @@ If demand grows beyond what the semaphore-based approach can handle, a proper qu #### Policy Management -`policy` stores external references only as the policy is fetched by Conforma at validation time, therefore Trustify does not cache policy content. +When the policy.policy_type is "Conforma", the initial only policy type supported, the `policy` is using external references only and therefore Trustify does not cache policy content. -The trade-off: validation always uses the latest policy version, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the configuration JSONB column and will be encrypted using AES crate; they are never logged. +Conforma fetches the policy at validation time from the git source specified in `policy.configuration.policy_ref`. -The policy commit hash/tag (`policy_version`) resolved at validation time are recorded in each result row, enabling reproducibility and audit. +The trade-off: validation always uses the latest policy content from the referenced branch or tag, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the `configuration` JSONB column and encrypted using the AES crate; they are never logged. + +The `policy_validation.policy_version` field records the policy commit hash or tag resolved from the `policy_ref` git source at validation time, enabling reproducibility and audit. +`policy_validation.conforma_version`, which tracks the Conforma CLI tool version number (e.g., `v0.8.83`). ### Futur work From 666f1d9caf2f117cfa9a4a418c573453e666ca03 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 10:18:04 +0200 Subject: [PATCH 59/77] Restore policy_validation.summary --- docs/adrs/00014-enterprise-contract-integration.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 973230258..d28f37387 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -388,13 +388,7 @@ sequenceDiagram - `processing_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' - `verification_status` (ENUM) - 'pending', 'pass', 'fail', 'error' - `results` (JSONB) - See model below -- `success` (BOOLEAN) - Overall pass/fail outcome (mirrors Conforma's top-level `success` field) -- `total` (SMALLINT) - Total number of checks evaluated -- `violations` (SMALLINT) - Count of checks with violation severity -- `warnings` (SMALLINT) - Count of checks with warning severity -- `successes` (SMALLINT) - Count of checks that passed -- `conforma_version`(VARCHAR) - Conforma version used (e.g. `"v0.8.83"`) -- `effective_time` (VARCHAR) - ISO 8601 timestamp of evaluation provided by Conforma +- `summary` (JSONB) - Total checks, passed, failed, warnings, see model below - `source_document_id` (VARCHAR) - File system or S3 path to detailed report - `start_time` (TIMESTAMP) - `end_time` (TIMESTAMP) From d837b27e82c9fa7e282b969204e49ea7d8c8c66e Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 10:19:57 +0200 Subject: [PATCH 60/77] Validation start/end time replaced with duration from conforma --- docs/adrs/00014-enterprise-contract-integration.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index d28f37387..3d6ff45d8 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -390,8 +390,6 @@ sequenceDiagram - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings, see model below - `source_document_id` (VARCHAR) - File system or S3 path to detailed report -- `start_time` (TIMESTAMP) -- `end_time` (TIMESTAMP) - `error_message` (TEXT) - Populated only on error status **`policy_validation.results` JSONB model:** From eed2ab519c9ed21b5ad6cbe8a8100725b4db82ed Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 10:29:50 +0200 Subject: [PATCH 61/77] Validation callback to use OIDC --- .../00014-enterprise-contract-integration.md | 44 +++++++++++++++---- 1 file changed, 36 insertions(+), 8 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 3d6ff45d8..82aedbda3 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -305,6 +305,7 @@ sequenceDiagram participant S3 as Object Storage participant Wrapper as Conforma Wrapper (HTTP) participant Conf as Conforma CLI + participant OIDC as OIDC Provider Note over Wrapper: Wrapper holds validation_id from the initial request @@ -320,7 +321,9 @@ sequenceDiagram end Wrapper->>Wrapper: Cleanup temp files - Wrapper->>API: POST /api/v2/ec/validation/{validation_id}/result {conforma JSON output} + Wrapper->>OIDC: POST /token (grant_type=client_credentials, client_id, client_secret) + OIDC-->>Wrapper: {access_token, expires_in} + Wrapper->>API: POST /api/v2/ec/validation/{validation_id}/result {conforma JSON output}
Authorization: Bearer API->>EP: dispatch callback EP->>VS: process_validation_result(validation_id, result) @@ -817,21 +820,22 @@ Download the full raw Conforma JSON report from storage. Callback endpoint used by the Conforma Wrapper to post the validation result back to Trustify after Conforma CLI execution completes. -This endpoint is not intended for end-user use. +This endpoint is not intended for end-user use. Because Trustify enforces OAuth 2.0 authentication on all API endpoints, the Conforma Wrapper must present a valid Bearer token obtained via the **OAuth 2.0 Client Credentials Grant** (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration) below). #### Request -| part | name | type | description | -| ---- | --------------- | -------- | --------------------------------------------------- | -| path | `validation_id` | `String` | ID of the validation (returned in the 202 response) | -| body | - | raw JSON | The raw Conforma CLI JSON output | +| part | name | type | description | +| ------ | --------------- | -------- | -------------------------------------------------------------------------------------- | +| path | `validation_id` | `String` | ID of the validation (returned in the 202 response) | +| header | `Authorization` | `String` | `Bearer ` — token obtained from the OIDC provider via Client Credentials | +| body | - | raw JSON | The raw Conforma CLI JSON output | #### Response - 204 - the result was accepted and persisted - 400 - if the request could not be understood or the JSON is malformed -- 401 - if the caller was not authenticated -- 403 - if the caller was not authorized +- 401 - if the caller was not authenticated (missing or invalid Bearer token) +- 403 - if the caller was not authorized (token lacks required scope/role) - 404 - if the validation ID was not found - 409 - if the validation already has a result (duplicate callback) @@ -845,6 +849,8 @@ POST /api/v1/validate # Validate uploaded SBOM file agains Accept an SBOM document and policy reference, spawn a Conforma CLI validation, and asynchronously post the result back to the Trustify callback endpoint. +The Conforma Wrapper must be pre-configured with OIDC client credentials so it can authenticate to Trustify when posting results back (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration)). + #### Request | part | name | type | description | @@ -901,6 +907,28 @@ modules/policy/ ### Technical Considerations +#### Conforma Wrapper OIDC Configuration + +The Conforma Wrapper acts as a service client that must authenticate to Trustify when posting validation results back via the callback endpoint. It uses the **OAuth 2.0 Client Credentials Grant** — a standard machine-to-machine flow where the Wrapper exchanges its client credentials for an access token from the OIDC provider. + +The following environment variables (or equivalent configuration) must be set at deployment time: + +| Variable | Required | Description | +| --------------------------------- | -------- | --------------------------------------------------------------------------------------------------- | +| `TRUSTIFY_OIDC_TOKEN_ENDPOINT` | yes | OIDC token endpoint URL (e.g. `https://keycloak.example.com/realms/trustify/protocol/openid-connect/token`) | +| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper | +| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client | +| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request (defaults to provider default; set if Trustify requires a specific scope) | + +**Token lifecycle**: The Wrapper should cache the access token and refresh it proactively before expiry (using the `expires_in` value from the token response). This avoids a token request on every callback and handles clock skew by refreshing with a safety margin (e.g. 30 seconds before expiry). + +**Failure handling**: If the token request fails (OIDC provider unreachable, invalid credentials), the Wrapper cannot deliver the callback. The validation remains in `in_progress` state until Trustify's timeout mechanism marks it as `failed`. The Wrapper should log the token acquisition error and may retry with exponential backoff before giving up. + +**Security considerations**: +- The client secret must be stored securely (e.g. Kubernetes Secret, vault injection) and never logged. +- The OIDC client should be registered with minimal scopes/roles — only the permission required to call the callback endpoint (`policy:write` or equivalent). +- TLS is required for all OIDC token endpoint communication. + #### Conforma CLI Execution The Conforma Wrapper invokes Conforma CLI via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); SBOMs are written to a temp file and passed by path in order to avoid OOM issues as SBOM can be very large file they shouldn't not be transfered via STDIN stream. From 18096da721f3b8ca5307842f672c62a6d3e6b802 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 30 Mar 2026 10:51:51 +0200 Subject: [PATCH 62/77] Align data model --- .../00014-enterprise-contract-integration.md | 56 +++++++++---------- 1 file changed, 27 insertions(+), 29 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 82aedbda3..64a84b4ec 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -63,7 +63,7 @@ The `processing_status` "In Progress" state serves as a concurrency guard: if a What is stored where -- PostgreSQL: validation process state (`processing_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, processing_status, status, start_time. +- PostgreSQL: validation process state (`processing_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, processing_status. - Storage system: full raw Conforma JSON report, linked from the DB row via `source_document.id`. Keeps DB rows small while preserving audit completeness. - Not stored: the policy definitions themselves. policy stores references (URLs, OCI refs) that Conforma fetches at runtime. @@ -517,7 +517,7 @@ struct PolicyConfiguration { ``` ```rust -/// Validation result summary returned by the API +/// Validation result returned by the API #[derive(Serialize, Deserialize)] struct PolicyValidation { id: Uuid, @@ -526,33 +526,30 @@ struct PolicyValidation { processing_status: String, verification_status: String, #[serde(default, skip_serializing_if = "Option::is_none")] - success: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - total: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - violations: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - warnings: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - successes: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - conforma_version: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - effective_time: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] results: Option>, #[serde(default, skip_serializing_if = "Option::is_none")] - source_document_id: Option, - start_time: String, - #[serde(default, skip_serializing_if = "Option::is_none")] - end_time: Option, + summary: Option, #[serde(default, skip_serializing_if = "Option::is_none")] - policy_version: Option, + source_document_id: Option, #[serde(default, skip_serializing_if = "Option::is_none")] error_message: Option, } ``` +```rust +/// Summary of a validation execution (stored as JSONB) +#[derive(Serialize, Deserialize)] +struct PolicyValidationSummary { + success: bool, + total: u16, + violations: u16, + warnings: u16, + successes: u16, + conforma_version: String, + effective_time: String, +} +``` + ```rust /// A single check result within a validation #[derive(Serialize, Deserialize)] @@ -760,7 +757,7 @@ Get the latest validation result for a given SBOM and policy pair. ### GET `/api/v2/policy/report/history` -Get the validation history for a given SBOM and policy pair, ordered by `start_time` descending. +Get the validation history for a given SBOM and policy pair, ordered by `sbom_id` descending. #### Request @@ -913,18 +910,19 @@ The Conforma Wrapper acts as a service client that must authenticate to Trustify The following environment variables (or equivalent configuration) must be set at deployment time: -| Variable | Required | Description | -| --------------------------------- | -------- | --------------------------------------------------------------------------------------------------- | -| `TRUSTIFY_OIDC_TOKEN_ENDPOINT` | yes | OIDC token endpoint URL (e.g. `https://keycloak.example.com/realms/trustify/protocol/openid-connect/token`) | -| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper | -| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client | -| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request (defaults to provider default; set if Trustify requires a specific scope) | +| Variable | Required | Description | +| ------------------------------ | -------- | ----------------------------------------------------------------------------------------------------------- | +| `TRUSTIFY_OIDC_TOKEN_ENDPOINT` | yes | OIDC token endpoint URL (e.g. `https://keycloak.example.com/realms/trustify/protocol/openid-connect/token`) | +| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper | +| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client | +| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request (defaults to provider default; set if Trustify requires a specific scope) | **Token lifecycle**: The Wrapper should cache the access token and refresh it proactively before expiry (using the `expires_in` value from the token response). This avoids a token request on every callback and handles clock skew by refreshing with a safety margin (e.g. 30 seconds before expiry). **Failure handling**: If the token request fails (OIDC provider unreachable, invalid credentials), the Wrapper cannot deliver the callback. The validation remains in `in_progress` state until Trustify's timeout mechanism marks it as `failed`. The Wrapper should log the token acquisition error and may retry with exponential backoff before giving up. **Security considerations**: + - The client secret must be stored securely (e.g. Kubernetes Secret, vault injection) and never logged. - The OIDC client should be registered with minimal scopes/roles — only the permission required to call the callback endpoint (`policy:write` or equivalent). - TLS is required for all OIDC token endpoint communication. @@ -953,7 +951,7 @@ Conforma fetches the policy at validation time from the git source specified in The trade-off: validation always uses the latest policy content from the referenced branch or tag, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the `configuration` JSONB column and encrypted using the AES crate; they are never logged. The `policy_validation.policy_version` field records the policy commit hash or tag resolved from the `policy_ref` git source at validation time, enabling reproducibility and audit. -`policy_validation.conforma_version`, which tracks the Conforma CLI tool version number (e.g., `v0.8.83`). +`policy_validation.summary.onforma_version`, which tracks the Conforma CLI tool version number (e.g., `v0.8.83`). ### Futur work From bbd4f11acd1dd8e88c0bca669c1e463894c3090e Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 7 Apr 2026 13:33:46 +0200 Subject: [PATCH 63/77] Typo --- docs/adrs/00014-enterprise-contract-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 64a84b4ec..e8ff76d63 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -951,7 +951,7 @@ Conforma fetches the policy at validation time from the git source specified in The trade-off: validation always uses the latest policy content from the referenced branch or tag, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the `configuration` JSONB column and encrypted using the AES crate; they are never logged. The `policy_validation.policy_version` field records the policy commit hash or tag resolved from the `policy_ref` git source at validation time, enabling reproducibility and audit. -`policy_validation.summary.onforma_version`, which tracks the Conforma CLI tool version number (e.g., `v0.8.83`). +`policy_validation.summary.conforma_version`, which tracks the Conforma CLI tool version number (e.g., `v0.8.83`). ### Futur work From d8b705958384dad9bfe0cf1cea784cc792d12b9f Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 7 Apr 2026 14:46:17 +0200 Subject: [PATCH 64/77] Move up wrapper consequences --- .../00014-enterprise-contract-integration.md | 42 +++++++++++-------- 1 file changed, 25 insertions(+), 17 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index e8ff76d63..3939ba3b6 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -31,11 +31,25 @@ Trustify stores information to identify (id, name, URL) of Policies. A default Policy is defined at the application level (global policy) which is used for validation when an SBOM does not have any Policy explicitly attached to it. Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. -A Conforma Wrapper (HTTP service) acts as a proxy between Trustify's Policy Verifier service and Conforma CLI. -Each SBOM + policy pair has two validation states . +A Conforma Wrapper (HTTP service) will act as a proxy between Trustify's Policy Verifier service and Conforma CLI. -The validation process state of the Conforma Wrapper follows this lifecycle: +### The Conforma HTTP Wrapper + +Policy validation can be be very resource-intensive, especially for large SBOMs with thousands of packages, and it requires a dedicated environment and having the Conforma HTTP Wrapper running alongside the Conforma CLI provides : + +- **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. +- **Independent scaling** — The Conforma Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. +- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure as a terminal `processing_status` and remains fully operational; the user may manually queue a new validation. +- **Version independence** — The Conforma Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. + +### Validation process state + +Each SBOM + policy pair has two validation states + +The validation process state of the + +To track the progress of the external validation process through the Conforma HTTP Wrapper, the following states are used : - **Queued** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. - **In Progress** — the request has been submitted to Conforma Wrapper. @@ -52,16 +66,18 @@ stateDiagram-v2 Failed --> [*] ``` -The Policy validation outcome follows this lifecycle: +The `processing_status` "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. + +### Policy validation state + +The result of a Policy validation follows this lifecycle: - **Pending** — initial state, indicates no validation has been triggered yet for this SBOM against this policy. - **Fail** — Conforma validation found policy violations; violation details are linked. - **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. - **Error** — The Conforma validation has generated an error. -The `processing_status` "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. - -What is stored where +### What is stored where - PostgreSQL: validation process state (`processing_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, processing_status. - Storage system: full raw Conforma JSON report, linked from the DB row via `source_document.id`. Keeps DB rows small while preserving audit completeness. @@ -71,17 +87,9 @@ Storing full JSON in storage system rather than only a summary was chosen explic ## Consequences -### The Conforma Wrapper runs externally - -EC validation can be be very resource-intensive (especially for large SBOMs with thousands of packages) and it should not compete with Trustify. -A dedicated Conforma Wrapper running alonside EC instance (Conforma CLI, etc) provides : - -- **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. -- **Independent scaling** — The Conforma Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. -- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure as a terminal `processing_status` and remains fully operational; the user may manually queue a new validation. -- **Version independence** — The Conforma Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. +### Conforma HTTP Wrapper -The trade-off is added infrastructure complexity: the Conforma Wrapper must be deployed separatly with EC instance, monitored, and maintained as a separate component alongside the Conforma CLI binary. +It's deployed separatly alongside the Conforma CLI instance, monitored, and maintained as a separate component. In Kubernetes or standalone machine deployments, the Conforma Wrapper pod has its own resource requests/limits, independent of the Trustify pod. From 9edf977bd6535a34b75144b06446aa87c756bb1a Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 7 Apr 2026 15:26:27 +0200 Subject: [PATCH 65/77] State machine only for validation process --- .../00014-enterprise-contract-integration.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 3939ba3b6..423422639 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -72,7 +72,7 @@ The `processing_status` "In Progress" state serves as a concurrency guard: if a The result of a Policy validation follows this lifecycle: -- **Pending** — initial state, indicates no validation has been triggered yet for this SBOM against this policy. +- **Null** — initial state, indicates no validation has been triggered yet for this SBOM against this policy. - **Fail** — Conforma validation found policy violations; violation details are linked. - **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. - **Error** — The Conforma validation has generated an error. @@ -336,15 +336,15 @@ sequenceDiagram EP->>VS: process_validation_result(validation_id, result) alt Pass - VS->>DB: UPDATE policy_validation SET verification_status='completed', status='pass', results=[] + VS->>DB: UPDATE policy_validation SET status='completed', result='pass', results=[] VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET source_document.id = ? else Fail - VS->>DB: UPDATE policy_validation SET verification_status='completed', status='fail', results=json + VS->>DB: UPDATE policy_validation SET status='completed', result='fail', results=json VS->>S3: store_validation_report(result_id, full_json) VS->>DB: UPDATE SET source_document.id = ? else Error - VS->>DB: UPDATE policy_validation SET verification_status='failed', status='error', error_message=detail + VS->>DB: UPDATE policy_validation SET status='completed', result='error', error_message=detail Note over VS,DB: Failed is terminal. User may manually queue a new validation (new row). end ``` @@ -396,8 +396,8 @@ sequenceDiagram - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → policy) -- `processing_status` (ENUM) - 'queued', 'in_progress', 'completed', 'failed' -- `verification_status` (ENUM) - 'pending', 'pass', 'fail', 'error' +- `processing_status` (ENUM) - 'null', 'queued', 'in_progress', 'completed', 'failed' +- `result` (ENUM) - 'null', 'fail', 'pass' or 'error' - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings, see model below - `source_document_id` (VARCHAR) - File system or S3 path to detailed report @@ -531,8 +531,8 @@ struct PolicyValidation { id: Uuid, sbom_id: Uuid, policy_id: Uuid, - processing_status: String, - verification_status: String, + state: String, + result: String, #[serde(default, skip_serializing_if = "Option::is_none")] results: Option>, #[serde(default, skip_serializing_if = "Option::is_none")] @@ -780,7 +780,7 @@ Get the validation history for a given SBOM and policy pair, ordered by `sbom_id The following `q` parameters are supported: - `processing_status`: Filters by processing status (`queued`, `in_progress`, `completed`, `failed`). -- `verification_status`: Filters by verification status (`pending`, `pass`, `fail`, `error`). +- `result`: Filters by verification status (`pending`, `pass`, `fail`, `error`). #### Response From f9e5a51c1a99b44489af753d5e5b662d0c109470 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 7 Apr 2026 15:36:46 +0200 Subject: [PATCH 66/77] Adjust status --- docs/adrs/00014-enterprise-contract-integration.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 423422639..4893a6c18 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -40,7 +40,7 @@ Policy validation can be be very resource-intensive, especially for large SBOMs - **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. - **Independent scaling** — The Conforma Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. -- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure as a terminal `processing_status` and remains fully operational; the user may manually queue a new validation. +- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure as a terminal `status` and remains fully operational; the user may manually queue a new validation. - **Version independence** — The Conforma Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. ### Validation process state @@ -66,7 +66,7 @@ stateDiagram-v2 Failed --> [*] ``` -The `processing_status` "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. +The "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. ### Policy validation state @@ -79,7 +79,7 @@ The result of a Policy validation follows this lifecycle: ### What is stored where -- PostgreSQL: validation process state (`processing_status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, processing_status. +- PostgreSQL: validation process state (`status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, status. - Storage system: full raw Conforma JSON report, linked from the DB row via `source_document.id`. Keeps DB rows small while preserving audit completeness. - Not stored: the policy definitions themselves. policy stores references (URLs, OCI refs) that Conforma fetches at runtime. @@ -282,7 +282,7 @@ sequenceDiagram EP-->>API: 404 Not Found end - VS->>DB: SELECT * FROM policy_validation WHERE sbom_id = ? AND policy_id = ? AND processing_status IN ('queued', 'in_progress') + VS->>DB: SELECT * FROM policy_validation WHERE sbom_id = ? AND policy_id = ? AND status IN ('queued', 'in_progress') alt Validation already in progress VS-->>EP: 409 Conflict {existing job_id} EP-->>API: 409 Conflict @@ -295,7 +295,7 @@ sequenceDiagram VS->>Wrapper: POST /api/v1/validate {SBOM, policy_ref} Wrapper-->>VS: 202 Accepted {validation_id} - VS->>DB: INSERT policy_validation (processing_status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) + VS->>DB: INSERT policy_validation (status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) VS-->>EP: 202 Accepted {validation_id} EP-->>API: 202 Accepted {validation_id} API-->>User: 202 Accepted {validation_id} @@ -396,7 +396,7 @@ sequenceDiagram - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → policy) -- `processing_status` (ENUM) - 'null', 'queued', 'in_progress', 'completed', 'failed' +- `status` (ENUM) - 'null', 'queued', 'in_progress', 'completed', 'failed' - `result` (ENUM) - 'null', 'fail', 'pass' or 'error' - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings, see model below @@ -779,7 +779,7 @@ Get the validation history for a given SBOM and policy pair, ordered by `sbom_id The following `q` parameters are supported: -- `processing_status`: Filters by processing status (`queued`, `in_progress`, `completed`, `failed`). +- `status`: Filters by processing status (`queued`, `in_progress`, `completed`, `failed`). - `result`: Filters by verification status (`pending`, `pass`, `fail`, `error`). #### Response From 5dc99b7eb73a01f5ec94512ffd7f763df1361020 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Tue, 7 Apr 2026 15:38:09 +0200 Subject: [PATCH 67/77] Adjust state diagram --- docs/adrs/00014-enterprise-contract-integration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 4893a6c18..f2cc1654a 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -59,9 +59,9 @@ To track the progress of the external validation process through the Conforma HT ```mermaid stateDiagram-v2 [*] --> Queued : User triggers validation - Queued --> InProgress : Request submitted to\nConforma Wrapper + Queued --> InProgress : Request submitted to
Conforma Wrapper InProgress --> Completed : Outcome received - InProgress --> Failed : Execution error\n(crash, timeout, fetch failure) + InProgress --> Failed : Execution error
(crash, timeout, fetch failure) Completed --> [*] Failed --> [*] ``` From 1bc6b1f20c1c066f7f8ace3c9000f8006adfe69e Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 8 Apr 2026 16:51:13 +0200 Subject: [PATCH 68/77] Validation request via insert on conflict --- .../adrs/00014-enterprise-contract-integration.md | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index f2cc1654a..b276ba145 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -58,24 +58,15 @@ To track the progress of the external validation process through the Conforma HT ```mermaid stateDiagram-v2 - [*] --> Queued : User triggers validation + [*] --> Queued : User triggers validation
INSERT() ON CONFLICT DO NOTHING Queued --> InProgress : Request submitted to
Conforma Wrapper InProgress --> Completed : Outcome received InProgress --> Failed : Execution error
(crash, timeout, fetch failure) - Completed --> [*] + Completed --> [*] : Update validation result
(pass, fail or error) Failed --> [*] ``` -The "In Progress" state serves as a concurrency guard: if a validation is already running for a given SBOM + policy pair, subsequent requests are rejected (409 Conflict), preventing duplicate work. - -### Policy validation state - -The result of a Policy validation follows this lifecycle: - -- **Null** — initial state, indicates no validation has been triggered yet for this SBOM against this policy. -- **Fail** — Conforma validation found policy violations; violation details are linked. -- **Pass** — Conforma validation succeeded; the SBOM satisfies the policy. -- **Error** — The Conforma validation has generated an error. +The result of a Policy validation is updated only when the validation process is completed. ### What is stored where From ce8b6fd09e9da15c8778a2c97ee0f953fcdcbbe4 Mon Sep 17 00:00:00 2001 From: arpcv Date: Wed, 15 Apr 2026 10:18:04 +0200 Subject: [PATCH 69/77] Add policy_validation.error field, more ec replacements --- docs/adrs/00014-enterprise-contract-integration.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index b276ba145..467c40584 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -40,8 +40,8 @@ Policy validation can be be very resource-intensive, especially for large SBOMs - **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. - **Independent scaling** — The Conforma Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. -- **Failure containment** — An EC instance crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper. Trustify records the failure as a terminal `status` and remains fully operational; the user may manually queue a new validation. -- **Version independence** — The Conforma Wrapper and EC instance (Conforma CLI) can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. +- **Failure containment** — An Conforma CLI crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper which will propagate back to Trustify the failure; This is a terminal state where the user will need to queue a new validation. In case the wrapper crashed, a timeout period will force Trustify to change the state of the queued validations to "Failed". +- **Version independence** — The Conforma Wrapper and Conforma CLI can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. ### Validation process state @@ -388,6 +388,7 @@ sequenceDiagram - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → policy) - `status` (ENUM) - 'null', 'queued', 'in_progress', 'completed', 'failed' +- `error`(TEXT) - Error message - `result` (ENUM) - 'null', 'fail', 'pass' or 'error' - `results` (JSONB) - See model below - `summary` (JSONB) - Total checks, passed, failed, warnings, see model below From 217412368e31cfefa050b811f11437fd1b73347f Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 15 Apr 2026 11:38:29 +0200 Subject: [PATCH 70/77] Validation is per policy id --- .../00014-enterprise-contract-integration.md | 64 +++++++++---------- 1 file changed, 31 insertions(+), 33 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 467c40584..38a598698 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -106,7 +106,7 @@ Conforma is not available as WASM and would require major upstream changes. A Redis/RabbitMQ queue would improve retry handling and priority management; implement if the 429-based rejection approach proves insufficient under real load. -## The solution +## Details ### System Architecture @@ -164,8 +164,8 @@ C4Container Rel(user, api, "Views compliance status", "HTTP API") Rel(webui, api, "API calls", "JSON/HTTP API") Rel(api, policyValidationModule, "Triggers validation", "Function call") - Rel(policyValidationModule, conformaWrapper, "POST /validation", "HTTP API formData") - Rel(conformaWrapper, api, "POST /validation/{id}/result", "HTTP API") + Rel(policyValidationModule, conformaWrapper, "POST /api/v1/validate", "HTTP API formData") + Rel(conformaWrapper, api, "POST /api/v2/policy/{id}/validation/{validation_id}/result", "HTTP API") Rel(conformaWrapper, conforma, "ec validate input {SBOM} {policy}", "Spawned command") Rel(policyValidationModule, postgres, "Saves validation
results", "SQL") Rel(policyValidationModule, storage, "Stores EC reports", "Function call") @@ -218,13 +218,13 @@ C4Component System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") } - Rel(api, policyEndpoints, "POST /ec/validate,\nGET /ec/report", "JSON/HTTPS") + Rel(api, policyEndpoints, "POST /api/v2/policy/{id}/validation,\nGET /api/v2/policy/{id}/validation/report,\nGET /api/v2/policy/{id}/validation/report/history,\nGET /api/v2/policy/{id}/validation/report/{result_id},\nPOST /api/v2/policy/{id}/validation/{validation_id}/result", "JSON/HTTPS") Rel(policyEndpoints, policyVeriferService, "validate_sbom() / get_ec_report()", "Function call") Rel(policyVeriferService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT policy", "SQL") - Rel(policyVeriferService, conformaWrapper, "POST /api/v1/validation → returns {id}", "HTTP") + Rel(policyVeriferService, conformaWrapper, "POST /api/v1/validate → returns {id}", "HTTP") Rel(conformaWrapper, conforma, "ec validate", "Process spawn") - Rel(conformaWrapper, api, "POST /api/v2/ec/validation/{validation_id}/result", "JSON/HTTPS") + Rel(conformaWrapper, api, "POST /api/v2/policy/{id}/validation/{validation_id}/result", "JSON/HTTPS") Rel(policyVeriferService, resultParser, "parse_output()", "Function call") Rel(policyVeriferService, resultPersistence, "save_results()", "Function call") Rel(resultPersistence, postgres, "INSERT policy_validation", "SQL") @@ -254,7 +254,7 @@ sequenceDiagram participant S3 as Object Storage participant Wrapper as Conforma Wrapper (HTTP) - User->>API: POST /api/v2/ec/validate (multipart form: sbom_id, policy_id) + User->>API: POST /api/v2/policy//validation?sbom_id= API->>EP: dispatch request EP->>VS: validate_sbom(sbom_id, policy_id) @@ -322,7 +322,7 @@ sequenceDiagram Wrapper->>Wrapper: Cleanup temp files Wrapper->>OIDC: POST /token (grant_type=client_credentials, client_id, client_secret) OIDC-->>Wrapper: {access_token, expires_in} - Wrapper->>API: POST /api/v2/ec/validation/{validation_id}/result {conforma JSON output}
Authorization: Bearer + Wrapper->>API: POST /api/v2/policy/{id}/validation/{validation_id}/result {conforma JSON output}
Authorization: Bearer API->>EP: dispatch callback EP->>VS: process_validation_result(validation_id, result) @@ -701,18 +701,17 @@ Deleting a policy will fail if there are validation results referencing it. - 409 - if the policy has associated validation results - 412 - if the `IfMatch` header was present, but its value didn't match the stored revision -### POST `/api/v2/policy/validate` +### POST `/api/v2/policy/{id}/validation` -Trigger a policy validation for a given SBOM and policy pair. The validation is performed asynchronously by the Conforma Wrapper; a `validation_id` is returned immediately. +Trigger policy validation for a given SBOM. The validation is performed asynchronously by the Conforma Wrapper; a `validation_id` is returned immediately. If a validation is already in progress for the same SBOM + policy pair, the request is rejected with 409 Conflict. #### Request -| part | name | type | description | -| ----- | ----------- | -------- | ----------------------------------------------------------------- | -| query | `sbom_id` | `String` | ID of the SBOM to validate | -| query | `policy_id` | `String` | ID of the policy to validate against (omit to use default policy) | +| part | name | type | description | +| ----- | --------- | -------- | -------------------------- | +| query | `sbom_id` | `String` | ID of the SBOM to validate | #### Response @@ -732,16 +731,15 @@ If a validation is already in progress for the same SBOM + policy pair, the requ - 409 - if a validation is already in progress for this SBOM + policy pair - 429 - if the Conforma Wrapper has reached its concurrency limit -### GET `/api/v2/policy/report` +### GET `/api/v2/policy/{id}/validation/report` -Get the latest validation result for a given SBOM and policy pair. +Get the latest validation result for a given SBOM. #### Request -| part | name | type | description | -| ----- | ----------- | -------- | --------------------------------------------- | -| query | `sbom_id` | `String` | ID of the SBOM | -| query | `policy_id` | `String` | ID of the policy (omit to use default policy) | +| part | name | type | description | +| ----- | --------- | -------- | -------------- | +| query | `sbom_id` | `String` | ID of the SBOM | #### Response @@ -755,19 +753,18 @@ Get the latest validation result for a given SBOM and policy pair. - 403 - if the user was authenticated but not authorized - 404 - if the SBOM or policy was not found, or no validation has been performed yet -### GET `/api/v2/policy/report/history` +### GET `/api/v2/policy/{id}/validation/report/history` -Get the validation history for a given SBOM and policy pair, ordered by `sbom_id` descending. +Get the validation history for a given SBOM (newest first; typically ordered by validation row creation time or id descending). #### Request -| part | name | type | description | -| ----- | ----------- | ---------- | ------------------------------------------------------- | -| query | `sbom_id` | `String` | ID of the SBOM | -| query | `policy_id` | `String` | ID of the policy (omit to use default policy) | -| query | `q` | "q" string | "q style" query string | -| query | `limit` | u64 | Maximum number of items to return | -| query | `offset` | u64 | Initial items to skip before actually returning results | +| part | name | type | description | +| ----- | --------- | ---------- | ------------------------------------------------------- | +| query | `sbom_id` | `String` | ID of the SBOM | +| query | `q` | "q" string | "q style" query string | +| query | `limit` | u64 | Maximum number of items to return | +| query | `offset` | u64 | Initial items to skip before actually returning results | The following `q` parameters are supported: @@ -790,7 +787,7 @@ The following `q` parameters are supported: - 403 - if the user was authenticated but not authorized - 404 - if the SBOM or policy was not found -### GET `/api/v2/policy/report/{result_id}` +### GET `/api/v2/policy/{id}/validation/report/{result_id}` Download the full raw Conforma JSON report from storage. @@ -813,7 +810,7 @@ Download the full raw Conforma JSON report from storage. - 403 - if the user was authenticated but not authorized - 404 - if the validation result or report was not found -### POST `/api/v2/policy/validation/{validation_id}/result` +### POST `/api/v2/policy/{id}/validation/{validation_id}/result` Callback endpoint used by the Conforma Wrapper to post the validation result back to Trustify after Conforma CLI execution completes. @@ -823,6 +820,7 @@ This endpoint is not intended for end-user use. Because Trustify enforces OAuth | part | name | type | description | | ------ | --------------- | -------- | -------------------------------------------------------------------------------------- | +| path | `id` | `String` | Policy id; must match the policy under which the validation was started | | path | `validation_id` | `String` | ID of the validation (returned in the 202 response) | | header | `Authorization` | `String` | `Bearer ` — token obtained from the OIDC provider via Client Credentials | | body | - | raw JSON | The raw Conforma CLI JSON output | @@ -833,7 +831,7 @@ This endpoint is not intended for end-user use. Because Trustify enforces OAuth - 400 - if the request could not be understood or the JSON is malformed - 401 - if the caller was not authenticated (missing or invalid Bearer token) - 403 - if the caller was not authorized (token lacks required scope/role) -- 404 - if the validation ID was not found +- 404 - if the policy or validation ID was not found, or the validation does not belong to this policy - 409 - if the validation already has a result (duplicate callback) ## Conforma Wrapper API Endpoints @@ -854,7 +852,7 @@ The Conforma Wrapper must be pre-configured with OIDC client credentials so it c | --------- | -------------- | -------- | -------------------------------------------------------------------------- | | multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | | multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | -| multipart | `callback_url` | `String` | Trustify callback URL (`/api/v2/policy/validation/{validation_id}/result`) | +| multipart | `callback_url` | `String` | Trustify callback URL (`/api/v2/policy/{policy_id}/validation/{validation_id}/result`) | | multipart | `extra_args` | `String` | Additional CLI flags forwarded to Conforma (optional, JSON-encoded array) | #### Response From 19997d8e6bf93cd4b56fd5ce2dd2131a9ad4dfeb Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Wed, 15 Apr 2026 14:35:39 +0200 Subject: [PATCH 71/77] Conforma CLI: No OIDC yet --- docs/adrs/00014-enterprise-contract-integration.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 38a598698..332baf782 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -320,8 +320,6 @@ sequenceDiagram end Wrapper->>Wrapper: Cleanup temp files - Wrapper->>OIDC: POST /token (grant_type=client_credentials, client_id, client_secret) - OIDC-->>Wrapper: {access_token, expires_in} Wrapper->>API: POST /api/v2/policy/{id}/validation/{validation_id}/result {conforma JSON output}
Authorization: Bearer API->>EP: dispatch callback EP->>VS: process_validation_result(validation_id, result) @@ -820,7 +818,7 @@ This endpoint is not intended for end-user use. Because Trustify enforces OAuth | part | name | type | description | | ------ | --------------- | -------- | -------------------------------------------------------------------------------------- | -| path | `id` | `String` | Policy id; must match the policy under which the validation was started | +| path | `id` | `String` | Policy id; must match the policy under which the validation was started | | path | `validation_id` | `String` | ID of the validation (returned in the 202 response) | | header | `Authorization` | `String` | `Bearer ` — token obtained from the OIDC provider via Client Credentials | | body | - | raw JSON | The raw Conforma CLI JSON output | @@ -848,12 +846,12 @@ The Conforma Wrapper must be pre-configured with OIDC client credentials so it c #### Request -| part | name | type | description | -| --------- | -------------- | -------- | -------------------------------------------------------------------------- | -| multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | -| multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | +| part | name | type | description | +| --------- | -------------- | -------- | -------------------------------------------------------------------------------------- | +| multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | +| multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | | multipart | `callback_url` | `String` | Trustify callback URL (`/api/v2/policy/{policy_id}/validation/{validation_id}/result`) | -| multipart | `extra_args` | `String` | Additional CLI flags forwarded to Conforma (optional, JSON-encoded array) | +| multipart | `extra_args` | `String` | Additional CLI flags forwarded to Conforma (optional, JSON-encoded array) | #### Response From 2650966e220681c219ae8e9834e2807388bbdfec Mon Sep 17 00:00:00 2001 From: arpcv Date: Thu, 16 Apr 2026 15:33:10 +0200 Subject: [PATCH 72/77] Refactor common summary fields; Adjust data model implementation --- .../00014-enterprise-contract-integration.md | 113 +++++++++++------- 1 file changed, 69 insertions(+), 44 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 332baf782..d5c2c4063 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -389,7 +389,13 @@ sequenceDiagram - `error`(TEXT) - Error message - `result` (ENUM) - 'null', 'fail', 'pass' or 'error' - `results` (JSONB) - See model below -- `summary` (JSONB) - Total checks, passed, failed, warnings, see model below +- `success` (BOOL) - Overall pass/fail outcome (mirrors Conforma's top-level `success` field) +- `total` (NUMBER) - Total number of checks evaluated +- `violations`(NUMBER) - Count of checks with violation severity +- `warnings` (NUMBER) - Count of checks with warning severity +- `successes` (NUMBER) - Count of checks that passed +- `type_metadata` (JSONB) - Policy validator specific data +- `validation_time` (DATETIME) - Evaluation duration - `source_document_id` (VARCHAR) - File system or S3 path to detailed report - `error_message` (TEXT) - Populated only on error status @@ -441,34 +447,29 @@ sequenceDiagram ] ``` -**`policy_validation.summary` JSONB model:** +**`policy_validation.type_metadata` JSONB model in the Conforma case:** -| Field | Type | Required | Description | -| ------------------ | ------- | -------- | ------------------------------------------------------------------------ | -| `success` | boolean | yes | Overall pass/fail outcome (mirrors Conforma's top-level `success` field) | -| `total` | integer | yes | Total number of checks evaluated | -| `violations` | integer | yes | Count of checks with violation severity | -| `warnings` | integer | yes | Count of checks with warning severity | -| `successes` | integer | yes | Count of checks that passed | -| `conforma_version` | string | yes | Version of Conforma CLI (e.g. `"v0.8.83"`) | -| `effective_time` | string | yes | ISO 8601 timestamp of evaluation provided by Conforma | +| Field | Type | Required | Description | +| ------------------ | ------ | -------- | ------------------------------------------ | +| `conforma_version` | string | yes | Version of Conforma CLI (e.g. `"v0.8.83"`) | -`policy_validation.summary` example: +`policy_validation.type_metadata` example: ```json { - "success": false, - "total": 3, - "violations": 1, - "warnings": 1, - "successes": 1, - "conforma_version": "v0.8.83", - "effective_time": "2026-03-03T14:36:55.807826709Z" + "conforma_version": "v0.8.83" } ``` #### Data Model Implementation +```rust +enum ValidatorKind { + Null, + Conforma, +} +``` + ```rust /// The policy reference information #[derive(Serialize, Deserialize)] @@ -477,8 +478,10 @@ struct Policy { name: String, #[serde(default, skip_serializing_if = "Option::is_none")] description: Option, - policy_type: String, - configuration: PolicyConfiguration, + policy_type: ValidatorKind, + configuration: serde_json::Value, + /// Conditional updates compare this revision (also exposed as `ETag` on GET). + revision: Uuid, } ``` @@ -489,8 +492,20 @@ struct PolicyRequest { name: String, #[serde(default, skip_serializing_if = "Option::is_none")] description: Option, - policy_type: String, - configuration: PolicyConfiguration, + policy_type: ValidatorKind, + configuration: serde_json::Value, +} +``` + +```rust +/// Credentials for private policy repos (`policy.configuration.auth`) +#[derive(Serialize, Deserialize)] +struct PolicyAuth { + /// `"token"`, `"ssh_key"`, or `"none"` + #[serde(rename = "type")] + auth_type: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + token_encrypted: Option, } ``` @@ -515,18 +530,42 @@ struct PolicyConfiguration { ``` ```rust -/// Validation result returned by the API +/// Conforma-specific `policy_validation.type_metadata` +#[derive(Serialize, Deserialize)] +struct ConformaTypeMetadata { + conforma_version: String, +} +``` + +```rust +/// One row per validation execution (`policy_validation`) #[derive(Serialize, Deserialize)] struct PolicyValidation { - id: Uuid, - sbom_id: Uuid, - policy_id: Uuid, - state: String, + id: String, + sbom_id: String, + policy_id: String, + /// Lifecycle: `'null'`, `'queued'`, `'in_progress'`, `'completed'`, `'failed'` + status: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + error: Option, + /// Outcome: `'null'`, `'fail'`, `'pass'`, `'error'` result: String, #[serde(default, skip_serializing_if = "Option::is_none")] results: Option>, #[serde(default, skip_serializing_if = "Option::is_none")] - summary: Option, + success: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + total: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + violations: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + warnings: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + successes: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + type_metadata: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + validation_time: Option, #[serde(default, skip_serializing_if = "Option::is_none")] source_document_id: Option, #[serde(default, skip_serializing_if = "Option::is_none")] @@ -535,21 +574,7 @@ struct PolicyValidation { ``` ```rust -/// Summary of a validation execution (stored as JSONB) -#[derive(Serialize, Deserialize)] -struct PolicyValidationSummary { - success: bool, - total: u16, - violations: u16, - warnings: u16, - successes: u16, - conforma_version: String, - effective_time: String, -} -``` - -```rust -/// A single check result within a validation +/// A single check result within `policy_validation.results` #[derive(Serialize, Deserialize)] struct PolicyValidationResult { severity: String, From fdf5a274c93ebe2375a8341b478b91dd7d970be8 Mon Sep 17 00:00:00 2001 From: arpcv Date: Thu, 16 Apr 2026 15:38:05 +0200 Subject: [PATCH 73/77] Use Option --- .../00014-enterprise-contract-integration.md | 33 ++++++++++--------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index d5c2c4063..42f7e40dc 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -548,24 +548,27 @@ struct PolicyValidation { status: String, #[serde(default, skip_serializing_if = "Option::is_none")] error: Option, - /// Outcome: `'null'`, `'fail'`, `'pass'`, `'error'` - result: String, - #[serde(default, skip_serializing_if = "Option::is_none")] - results: Option>, - #[serde(default, skip_serializing_if = "Option::is_none")] - success: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - total: Option, + /// Present once `status` is `'completed'` or `'failed'` #[serde(default, skip_serializing_if = "Option::is_none")] - violations: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - warnings: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - successes: Option, + outcome: Option, +} +``` + +```rust +/// Outcome produced when a validation finishes (all-or-nothing) +#[derive(Serialize, Deserialize)] +struct ValidationOutcome { + /// `'fail'`, `'pass'`, `'error'` + result: String, + results: Vec, + success: bool, + total: u32, + violations: u32, + warnings: u32, + successes: u32, #[serde(default, skip_serializing_if = "Option::is_none")] type_metadata: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - validation_time: Option, + validation_time: String, #[serde(default, skip_serializing_if = "Option::is_none")] source_document_id: Option, #[serde(default, skip_serializing_if = "Option::is_none")] From d45712dc76ff925f13fa27c574f99eb88e41c327 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 16 Apr 2026 15:56:28 +0200 Subject: [PATCH 74/77] Severities: Used full fledged enum discrimantor --- .../00014-enterprise-contract-integration.md | 23 +++++++++++++++---- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 42f7e40dc..60853ba93 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -474,7 +474,7 @@ enum ValidatorKind { /// The policy reference information #[derive(Serialize, Deserialize)] struct Policy { - id: Uuid, + id: String, name: String, #[serde(default, skip_serializing_if = "Option::is_none")] description: Option, @@ -579,10 +579,23 @@ struct ValidationOutcome { ```rust /// A single check result within `policy_validation.results` #[derive(Serialize, Deserialize)] -struct PolicyValidationResult { - severity: String, - msg: String, - metadata: PolicyValidationResultMetadata, +#[serde(tag = "severity")] +enum PolicyValidationResult { + #[serde(rename = "violation")] + Violation { + msg: String, + metadata: PolicyValidationResultMetadata, + }, + #[serde(rename = "warning")] + Warning { + msg: String, + metadata: PolicyValidationResultMetadata, + }, + #[serde(rename = "success")] + Success { + msg: String, + metadata: PolicyValidationResultMetadata, + }, } ``` From 734724262fb9a97c9f1c38f50ee6fa17dd7d8c04 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Thu, 16 Apr 2026 18:04:45 +0200 Subject: [PATCH 75/77] Endpoints: Align format; Move module; Details auth and securities --- .../00014-enterprise-contract-integration.md | 172 ++++++++++++++---- 1 file changed, 133 insertions(+), 39 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index 60853ba93..ce29ae34e 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -354,9 +354,9 @@ sequenceDiagram | Field | Type | Required | Description | | ---------------------- | -------- | --------------- | -------------------------------------------------------------------------------- | | `policy_ref` | string | yes | Policy source URL, e.g. `"git://[URL]?ref=[BRANCH OR TAG]"` | -| `auth` | object | no | Credentials for private repos; sensitive values encrypted via AES (never logged) | +| `auth` | object | no | Credentials for private repos; sensitive values encrypted via `ring::aead` AES-256-GCM (never logged) | | `auth.type` | string | yes (if `auth`) | `"token"`, `"ssh_key"`, or `"none"` | -| `auth.token_encrypted` | string | no | AES-encrypted bearer/PAT token, prefixed with encryption scheme | +| `auth.token_encrypted` | string | no | AES-256-GCM encrypted bearer/PAT token, prefixed with encryption scheme | | `policy_paths` | string[] | no | Sub-paths within the repo to evaluate (maps to Conforma `--policy` source paths) | | `exclude` | string[] | no | Rule codes to skip during validation | | `include` | string[] | no | If non-empty, only these rule codes are evaluated | @@ -370,7 +370,7 @@ sequenceDiagram "policy_ref": "git://github.com/org/policy-repo?ref=main", "auth": { "type": "token", - "token_encrypted": "AES256:" + "token_encrypted": "AES-256-GCM::" }, "policy_paths": ["policy/lib", "policy/release"], "exclude": ["hello_world.minimal_packages"], @@ -396,7 +396,7 @@ sequenceDiagram - `successes` (NUMBER) - Count of checks that passed - `type_metadata` (JSONB) - Policy validator specific data - `validation_time` (DATETIME) - Evaluation duration -- `source_document_id` (VARCHAR) - File system or S3 path to detailed report +- `source_document_id` (VARCHAR) - File system or S3 id of the detailed report - `error_message` (TEXT) - Populated only on error status **`policy_validation.results` JSONB model:** @@ -611,6 +611,58 @@ struct PolicyValidationResultMetadata { } ``` +## Trustify API Endpoints + +``` +POST /api/v2/policy # Create a new policy reference +GET /api/v2/policy # List policy references +GET /api/v2/policy/{id} # Get a single policy reference +PUT /api/v2/policy/{id} # Update a policy reference +DELETE /api/v2/policy/{id} # Delete a policy reference +POST /api/v2/policy/{id}/validation # Trigger policy validation +GET /api/v2/policy/{id}/validation/report # Get latest validation result +GET /api/v2/policy/{id}/validation/report/history # Get validation history +GET /api/v2/policy/{id}/validation/report/{result_id} # Download full report +POST /api/v2/policy/{id}/validation/{validation_id}/result # Callback for validation result +``` + +### Permissions + +The policy module introduces the following permissions, following the existing Trustify CRUD convention: + +| Permission | Description | +| ---------------- | ------------------------------------------------------------------ | +| `create.policy` | Create policy references and trigger validations | +| `read.policy` | List/get policy references and read validation results and reports | +| `update.policy` | Update policy references and post validation results (callback) | +| `delete.policy` | Delete policy references | + +These permissions map to the default OIDC scope groups: + +| Scope | Permissions granted | +| ----------------- | ------------------------------ | +| `create:document` | `create.policy` | +| `read:document` | `read.policy` | +| `update:document` | `update.policy` | +| `delete:document` | `delete.policy` | + +Endpoint permission requirements: + +| Endpoint | Permission | +| ------------------------------------------------------------- | ---------------- | +| `POST /api/v2/policy` | `create.policy` | +| `GET /api/v2/policy` | `read.policy` | +| `GET /api/v2/policy/{id}` | `read.policy` | +| `PUT /api/v2/policy/{id}` | `update.policy` | +| `DELETE /api/v2/policy/{id}` | `delete.policy` | +| `POST /api/v2/policy/{id}/validation` | `create.policy` | +| `GET /api/v2/policy/{id}/validation/report` | `read.policy` | +| `GET /api/v2/policy/{id}/validation/report/history` | `read.policy` | +| `GET /api/v2/policy/{id}/validation/report/{result_id}` | `read.policy` | +| `POST /api/v2/policy/{id}/validation/{validation_id}/result` | `update.policy` | + +The Conforma Wrapper's OIDC client (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration)) should be registered with the minimal scope required — only `update:document` (granting `update.policy`) — so it can post results to the callback endpoint. + ### POST `/api/v2/policy` Create a new policy reference. @@ -875,24 +927,23 @@ This endpoint is not intended for end-user use. Because Trustify enforces OAuth ## Conforma Wrapper API Endpoints -``` -POST /api/v1/validate # Validate uploaded SBOM file against the provided Policy URL (multipart form) -``` - ### POST `/api/v1/validate` Accept an SBOM document and policy reference, spawn a Conforma CLI validation, and asynchronously post the result back to the Trustify callback endpoint. -The Conforma Wrapper must be pre-configured with OIDC client credentials so it can authenticate to Trustify when posting results back (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration)). +Because the Conforma Wrapper is a network-accessible service, it **must authenticate incoming requests** using the same OIDC provider that Trustify uses. Callers must present a valid Bearer token obtained from the shared OIDC provider. The Wrapper validates the token by fetching the provider's JWKS and verifying the signature, expiry, and issuer — exactly the same mechanism Trustify uses for its own endpoints (see `trustify-auth` / `Authenticator`). This ensures that only authorized Trustify instances (or other permitted clients) can trigger validations. + +The Conforma Wrapper must also be pre-configured with OIDC client credentials so it can authenticate to Trustify when posting results back (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration)). #### Request -| part | name | type | description | -| --------- | -------------- | -------- | -------------------------------------------------------------------------------------- | -| multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | -| multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | -| multipart | `callback_url` | `String` | Trustify callback URL (`/api/v2/policy/{policy_id}/validation/{validation_id}/result`) | -| multipart | `extra_args` | `String` | Additional CLI flags forwarded to Conforma (optional, JSON-encoded array) | +| part | name | type | description | +| --------- | --------------- | -------- | -------------------------------------------------------------------------------------- | +| header | `Authorization` | `String` | `Bearer ` — token obtained from the shared OIDC provider | +| multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | +| multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | +| multipart | `callback_url` | `String` | Trustify callback URL (`/api/v2/policy/{policy_id}/validation/{validation_id}/result`) | +| multipart | `extra_args` | `String` | Additional CLI flags forwarded to Conforma (optional, JSON-encoded array) | #### Response @@ -906,10 +957,11 @@ The Conforma Wrapper must be pre-configured with OIDC client credentials so it c ``` - 400 - if the request could not be understood or required fields are missing -- 401 - if the caller was not authenticated +- 401 - if the caller was not authenticated (missing or invalid Bearer token) +- 403 - if the caller was authenticated but not authorized (token lacks required scope/role) - 429 - if the concurrency semaphore is exhausted (too many concurrent validations) -### Trustify File Structure +## File Structure ``` modules/policy/ @@ -930,39 +982,68 @@ modules/policy/ │ │ └── result_parser.rs # Output parsing │ └── client/ │ └── conforma.rs # Conforma client adapter -└── conforma_wrapper - ├── build.rs - ├── Cargo.toml - └── src/ - ├── endpoints/ - │ └── mod.rs # REST endpoints - └── lib.rs +modules/conforma_wrapper +├── build.rs +├── Cargo.toml +└── src/ + ├── endpoints/ + │ └── mod.rs # REST endpoints + └── lib.rs ``` -### Technical Considerations +## Technical Considerations #### Conforma Wrapper OIDC Configuration -The Conforma Wrapper acts as a service client that must authenticate to Trustify when posting validation results back via the callback endpoint. It uses the **OAuth 2.0 Client Credentials Grant** — a standard machine-to-machine flow where the Wrapper exchanges its client credentials for an access token from the OIDC provider. +The Conforma Wrapper has two distinct OIDC responsibilities that share the **same OIDC provider** (e.g. the Keycloak realm used by Trustify): + +1. **Inbound authentication** — validate Bearer tokens on incoming requests to `/api/v1/validate`, ensuring only authorized callers (Trustify, CI systems, etc.) can trigger validations. +2. **Outbound authentication** — obtain an access token via the Client Credentials Grant to authenticate when posting results back to the Trustify callback endpoint. + +Because both directions use the same OIDC provider, the Wrapper only needs a single issuer URL. It fetches the provider's discovery document (`.well-known/openid-configuration`) once at startup to obtain the JWKS URI (for inbound token validation) and the token endpoint (for outbound token acquisition). + +Conversely, the **Conforma client adapter** in Trustify (`modules/policy/src/client/conforma.rs`) also acts as a server: it exposes the callback endpoint (`POST /api/v2/policy/{policy_id}/validation/{validation_id}/result`) that receives validation results from the Conforma Wrapper. This endpoint must be protected so that only the Wrapper — authenticated via its Client Credentials token — can post results. Because the callback endpoint is a standard Trustify REST endpoint, it is protected by Trustify's existing OIDC authentication middleware (`trustify-auth` / `Authenticator`), which validates the Bearer token the Wrapper obtained during its outbound token acquisition. The OIDC client registered for the Wrapper must be granted the minimum role or scope required to call this callback (e.g. `policy:write`), and Trustify must reject tokens that lack the necessary permission with 403 Forbidden. + +##### Environment Variables The following environment variables (or equivalent configuration) must be set at deployment time: -| Variable | Required | Description | -| ------------------------------ | -------- | ----------------------------------------------------------------------------------------------------------- | -| `TRUSTIFY_OIDC_TOKEN_ENDPOINT` | yes | OIDC token endpoint URL (e.g. `https://keycloak.example.com/realms/trustify/protocol/openid-connect/token`) | -| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper | -| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client | -| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request (defaults to provider default; set if Trustify requires a specific scope) | +| Variable | Required | Description | +| -------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| `TRUSTIFY_OIDC_ISSUER_URL` | yes | OIDC issuer URL (e.g. `https://keycloak.example.com/realms/trustify`). Used for discovery of JWKS, token endpoint, and issuer validation. | +| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper (used for both inbound audience validation and outbound token grant) | +| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client (used for the outbound Client Credentials Grant) | +| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request on outbound token grants (defaults to provider default; set if Trustify requires a specific scope) | +| `TRUSTIFY_OIDC_TLS_INSECURE` | no | Disable TLS certificate validation for the OIDC provider (default `false`; only for development) | +| `TRUSTIFY_OIDC_TLS_CA_CERTIFICATES` | no | Additional CA certificates to trust when communicating with the OIDC provider | + +##### Inbound Token Validation + +On startup the Wrapper fetches the JWKS from the OIDC provider's discovery endpoint and caches it. For each incoming request to `/api/v1/validate`: + +1. Extract the `Authorization: Bearer ` header. +2. Verify the JWT signature against the cached JWKS (refresh on cache miss for key rotation). +3. Validate standard claims: `iss` matches `TRUSTIFY_OIDC_ISSUER_URL`, token is not expired, `aud` or `azp` includes the expected client. +4. On failure return 401 Unauthorized. + +This follows the same validation logic that Trustify's own `Authenticator` uses (see `trustify-auth` crate), so the Wrapper can reuse or mirror that implementation. + +##### Outbound Token Acquisition + +For posting results back to Trustify the Wrapper uses the **OAuth 2.0 Client Credentials Grant**: **Token lifecycle**: The Wrapper should cache the access token and refresh it proactively before expiry (using the `expires_in` value from the token response). This avoids a token request on every callback and handles clock skew by refreshing with a safety margin (e.g. 30 seconds before expiry). -**Failure handling**: If the token request fails (OIDC provider unreachable, invalid credentials), the Wrapper cannot deliver the callback. The validation remains in `in_progress` state until Trustify's timeout mechanism marks it as `failed`. The Wrapper should log the token acquisition error and may retry with exponential backoff before giving up. +**Failure handling**: If the token request fails (OIDC provider unreachable, invalid credentials), the Wrapper cannot deliver the callback. The validation remains in `in_progress` state until Trustify's timeout mechanism marks it as `failed` (see below). The Wrapper should log the token acquisition error and may retry with exponential backoff before giving up. -**Security considerations**: +**Trustify timeout mechanism**: A background reaper task runs periodically inside the Policy Verifier service (default interval: 60 seconds). On each tick it queries for `policy_validation` rows whose `status` is `in_progress` and whose `updated_at` timestamp is older than the configured timeout (default: the policy's `timeout_seconds`, falling back to the global default of 5 minutes). Each stale row is transitioned to `status = 'failed'` with an `error` of `"Validation timed out"`. This covers every failure scenario where the Wrapper never delivers a callback — CLI hang, Wrapper crash, network partition, or token acquisition failure. The reaper uses an `UPDATE … WHERE status = 'in_progress' AND updated_at < NOW() - interval` query so it is idempotent and safe to run from multiple Trustify replicas concurrently. + +##### Security Considerations - The client secret must be stored securely (e.g. Kubernetes Secret, vault injection) and never logged. -- The OIDC client should be registered with minimal scopes/roles — only the permission required to call the callback endpoint (`policy:write` or equivalent). -- TLS is required for all OIDC token endpoint communication. +- The OIDC client registered for the Conforma Wrapper should be a **dedicated confidential client** (e.g. `conforma-wrapper`) in the OIDC provider, separate from clients used by human users or other services. This client should be granted a narrow, purpose-specific scope — `trustify:policy-callback` — that maps to the single permission needed to post validation results to the callback endpoint. Trustify's authorization middleware must enforce this scope on the callback route (`POST /api/v2/policy/{policy_id}/validation/{validation_id}/result`), rejecting tokens that lack it with 403 Forbidden. By using a dedicated client and scope rather than a broad role like `policy:write`, the blast radius of a compromised credential is limited to callback delivery only — the Wrapper cannot read, delete, or otherwise mutate policies or other Trustify resources. +- TLS is required for all OIDC communication (provider discovery, JWKS fetch, token endpoint). +- The Wrapper should reject tokens from unexpected issuers; the `TRUSTIFY_OIDC_ISSUER_URL` acts as an allowlist of exactly one trusted issuer. #### Conforma CLI Execution @@ -979,18 +1060,31 @@ Concurrency is controlled at two levels: If demand grows beyond what the semaphore-based approach can handle, a proper queue (Redis/RabbitMQ) is a deferred alternative (see _Batch Processing Queue_ in Alternatives Considered above). +##### Kubernetes Deployment + +When both Trustify and the Conforma Wrapper are deployed on a Kubernetes cluster, native K8s primitives can complement the application-level concurrency controls described above: + +- **Readiness and liveness probes** — The Conforma Wrapper exposes two health endpoints that Kubernetes uses to manage pod lifecycle and traffic routing: + - *Liveness* (`GET /healthz`) — returns 200 if the process is alive. Kubernetes restarts the pod if this probe fails (e.g. deadlock, unrecoverable panic). The check is lightweight: it confirms the HTTP server loop is responsive and, optionally, that the OIDC JWKS cache is populated. + - *Readiness* (`GET /readyz`) — returns 200 when the pod can accept new work, and 503 when it cannot. The readiness check is tied to the concurrency semaphore: when all permits are in use, the endpoint returns 503, signalling Kubernetes to remove the pod from the Service's endpoint list. New requests are then routed only to pods that still have capacity. Once a permit is released, the probe returns 200 and the pod re-enters the rotation. This provides cluster-level backpressure that is transparent to Trustify — the K8s Service load-balances across ready pods automatically, reducing the likelihood of 429 responses reaching the caller. A 429 is still returned as a last resort if a request arrives between a readiness check and semaphore exhaustion. +- **Horizontal Pod Autoscaler (HPA)** — The Wrapper Deployment can be configured with an HPA that scales replicas based on CPU/memory utilization or a custom metric such as the in-flight validation count exposed via a `/metrics` endpoint. Because the readiness probe already removes saturated pods from the Service, the HPA's scaling decisions and the readiness-driven traffic shifting work together: the HPA adds capacity while readiness prevents overload on existing pods. This allows the cluster to absorb demand spikes without requiring a centralized queue. +- **Resource limits and requests** — Each Wrapper pod should declare CPU and memory `requests` and `limits` that account for the peak resource usage of the CLI subprocess (the semaphore concurrency multiplied by the per-process footprint). This prevents a burst of validations from starving other workloads on the node. +- **NetworkPolicy** — A Kubernetes `NetworkPolicy` can restrict ingress to the Wrapper pods so that only Trustify pods (selected by label) are allowed to reach the `/api/v1/validate` endpoint. This provides a network-layer defense-in-depth on top of the OIDC-based authentication, ensuring that even if a valid token were leaked, it could not be used from outside the trusted namespace. +- **Service discovery** — Trustify references the Wrapper via its Kubernetes `Service` DNS name (e.g. `conforma-wrapper.trustify.svc.cluster.local`). This decouples Trustify from individual pod IPs and lets K8s load-balance requests across ready Wrapper replicas automatically. Combined with the readiness probe, Trustify never needs to be aware of individual pod capacity — the Service only routes to pods that have reported ready. +- **Secrets management** — The OIDC client secret and any policy-repo credentials should be mounted from Kubernetes `Secret` resources (or injected via an external secrets operator such as External Secrets or HashiCorp Vault Agent) rather than embedded in environment variable literals. This integrates with K8s RBAC to limit which pods and service accounts can access the sensitive material. + #### Policy Management When the policy.policy_type is "Conforma", the initial only policy type supported, the `policy` is using external references only and therefore Trustify does not cache policy content. Conforma fetches the policy at validation time from the git source specified in `policy.configuration.policy_ref`. -The trade-off: validation always uses the latest policy content from the referenced branch or tag, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the `configuration` JSONB column and encrypted using the AES crate; they are never logged. +The trade-off: validation always uses the latest policy content from the referenced branch or tag, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the `configuration` JSONB column and encrypted at rest using `ring::aead` (AES-256-GCM authenticated encryption); they are never logged. The `ring` crate is already a direct dependency of the project (used for digest hashing), so no new dependency is required. The `policy_validation.policy_version` field records the policy commit hash or tag resolved from the `policy_ref` git source at validation time, enabling reproducibility and audit. `policy_validation.summary.conforma_version`, which tracks the Conforma CLI tool version number (e.g., `v0.8.83`). -### Futur work +## Future Work #### Validation on SBOM upload @@ -1002,7 +1096,7 @@ Policy references are global (shared across all users) in this initial implement This is out of scope of this ADR. -### References +## References - [Enterprise Contract (Conforma) GitHub](https://github.com/enterprise-contract/ec-cli) - [Design Document](../design/enterprise-contract-integration.md) From 2ecaaa9f77b01b4785dae6cac3c29fb91b4347d2 Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 20 Apr 2026 14:37:03 +0200 Subject: [PATCH 76/77] No default policy --- .../00014-enterprise-contract-integration.md | 89 +++++++++---------- 1 file changed, 44 insertions(+), 45 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index ce29ae34e..deb35edf1 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -28,7 +28,6 @@ Users need the ability to: We will integrate Conforma into Trustify as a user triggered validation service by interacting with Conforma CLI. Validation is manually triggered — not automatic on SBOM upload. Trustify stores information to identify (id, name, URL) of Policies. -A default Policy is defined at the application level (global policy) which is used for validation when an SBOM does not have any Policy explicitly attached to it. Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. @@ -351,17 +350,17 @@ sequenceDiagram **`policy.configuration` JSONB model:** -| Field | Type | Required | Description | -| ---------------------- | -------- | --------------- | -------------------------------------------------------------------------------- | -| `policy_ref` | string | yes | Policy source URL, e.g. `"git://[URL]?ref=[BRANCH OR TAG]"` | +| Field | Type | Required | Description | +| ---------------------- | -------- | --------------- | ----------------------------------------------------------------------------------------------------- | +| `policy_ref` | string | yes | Policy source URL, e.g. `"git://[URL]?ref=[BRANCH OR TAG]"` | | `auth` | object | no | Credentials for private repos; sensitive values encrypted via `ring::aead` AES-256-GCM (never logged) | -| `auth.type` | string | yes (if `auth`) | `"token"`, `"ssh_key"`, or `"none"` | -| `auth.token_encrypted` | string | no | AES-256-GCM encrypted bearer/PAT token, prefixed with encryption scheme | -| `policy_paths` | string[] | no | Sub-paths within the repo to evaluate (maps to Conforma `--policy` source paths) | -| `exclude` | string[] | no | Rule codes to skip during validation | -| `include` | string[] | no | If non-empty, only these rule codes are evaluated | -| `timeout_seconds` | integer | no | Per-policy override of the default 5-minute execution timeout | -| `extra_args` | string[] | no | Additional CLI flags forwarded verbatim to Conforma | +| `auth.type` | string | yes (if `auth`) | `"token"`, `"ssh_key"`, or `"none"` | +| `auth.token_encrypted` | string | no | AES-256-GCM encrypted bearer/PAT token, prefixed with encryption scheme | +| `policy_paths` | string[] | no | Sub-paths within the repo to evaluate (maps to Conforma `--policy` source paths) | +| `exclude` | string[] | no | Rule codes to skip during validation | +| `include` | string[] | no | If non-empty, only these rule codes are evaluated | +| `timeout_seconds` | integer | no | Per-policy override of the default 5-minute execution timeout | +| `extra_args` | string[] | no | Additional CLI flags forwarded verbatim to Conforma | `policy.configuration` example : @@ -630,36 +629,36 @@ POST /api/v2/policy/{id}/validation/{validation_id}/result # Callbac The policy module introduces the following permissions, following the existing Trustify CRUD convention: -| Permission | Description | -| ---------------- | ------------------------------------------------------------------ | -| `create.policy` | Create policy references and trigger validations | -| `read.policy` | List/get policy references and read validation results and reports | -| `update.policy` | Update policy references and post validation results (callback) | -| `delete.policy` | Delete policy references | +| Permission | Description | +| --------------- | ------------------------------------------------------------------ | +| `create.policy` | Create policy references and trigger validations | +| `read.policy` | List/get policy references and read validation results and reports | +| `update.policy` | Update policy references and post validation results (callback) | +| `delete.policy` | Delete policy references | These permissions map to the default OIDC scope groups: -| Scope | Permissions granted | -| ----------------- | ------------------------------ | -| `create:document` | `create.policy` | -| `read:document` | `read.policy` | -| `update:document` | `update.policy` | -| `delete:document` | `delete.policy` | +| Scope | Permissions granted | +| ----------------- | ------------------- | +| `create:document` | `create.policy` | +| `read:document` | `read.policy` | +| `update:document` | `update.policy` | +| `delete:document` | `delete.policy` | Endpoint permission requirements: -| Endpoint | Permission | -| ------------------------------------------------------------- | ---------------- | -| `POST /api/v2/policy` | `create.policy` | -| `GET /api/v2/policy` | `read.policy` | -| `GET /api/v2/policy/{id}` | `read.policy` | -| `PUT /api/v2/policy/{id}` | `update.policy` | -| `DELETE /api/v2/policy/{id}` | `delete.policy` | -| `POST /api/v2/policy/{id}/validation` | `create.policy` | -| `GET /api/v2/policy/{id}/validation/report` | `read.policy` | -| `GET /api/v2/policy/{id}/validation/report/history` | `read.policy` | -| `GET /api/v2/policy/{id}/validation/report/{result_id}` | `read.policy` | -| `POST /api/v2/policy/{id}/validation/{validation_id}/result` | `update.policy` | +| Endpoint | Permission | +| ------------------------------------------------------------ | --------------- | +| `POST /api/v2/policy` | `create.policy` | +| `GET /api/v2/policy` | `read.policy` | +| `GET /api/v2/policy/{id}` | `read.policy` | +| `PUT /api/v2/policy/{id}` | `update.policy` | +| `DELETE /api/v2/policy/{id}` | `delete.policy` | +| `POST /api/v2/policy/{id}/validation` | `create.policy` | +| `GET /api/v2/policy/{id}/validation/report` | `read.policy` | +| `GET /api/v2/policy/{id}/validation/report/history` | `read.policy` | +| `GET /api/v2/policy/{id}/validation/report/{result_id}` | `read.policy` | +| `POST /api/v2/policy/{id}/validation/{validation_id}/result` | `update.policy` | The Conforma Wrapper's OIDC client (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration)) should be registered with the minimal scope required — only `update:document` (granting `update.policy`) — so it can post results to the callback endpoint. @@ -1008,14 +1007,14 @@ Conversely, the **Conforma client adapter** in Trustify (`modules/policy/src/cli The following environment variables (or equivalent configuration) must be set at deployment time: -| Variable | Required | Description | -| -------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -| `TRUSTIFY_OIDC_ISSUER_URL` | yes | OIDC issuer URL (e.g. `https://keycloak.example.com/realms/trustify`). Used for discovery of JWKS, token endpoint, and issuer validation. | -| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper (used for both inbound audience validation and outbound token grant) | -| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client (used for the outbound Client Credentials Grant) | -| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request on outbound token grants (defaults to provider default; set if Trustify requires a specific scope) | -| `TRUSTIFY_OIDC_TLS_INSECURE` | no | Disable TLS certificate validation for the OIDC provider (default `false`; only for development) | -| `TRUSTIFY_OIDC_TLS_CA_CERTIFICATES` | no | Additional CA certificates to trust when communicating with the OIDC provider | +| Variable | Required | Description | +| ----------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| `TRUSTIFY_OIDC_ISSUER_URL` | yes | OIDC issuer URL (e.g. `https://keycloak.example.com/realms/trustify`). Used for discovery of JWKS, token endpoint, and issuer validation. | +| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper (used for both inbound audience validation and outbound token grant) | +| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client (used for the outbound Client Credentials Grant) | +| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request on outbound token grants (defaults to provider default; set if Trustify requires a specific scope) | +| `TRUSTIFY_OIDC_TLS_INSECURE` | no | Disable TLS certificate validation for the OIDC provider (default `false`; only for development) | +| `TRUSTIFY_OIDC_TLS_CA_CERTIFICATES` | no | Additional CA certificates to trust when communicating with the OIDC provider | ##### Inbound Token Validation @@ -1065,8 +1064,8 @@ If demand grows beyond what the semaphore-based approach can handle, a proper qu When both Trustify and the Conforma Wrapper are deployed on a Kubernetes cluster, native K8s primitives can complement the application-level concurrency controls described above: - **Readiness and liveness probes** — The Conforma Wrapper exposes two health endpoints that Kubernetes uses to manage pod lifecycle and traffic routing: - - *Liveness* (`GET /healthz`) — returns 200 if the process is alive. Kubernetes restarts the pod if this probe fails (e.g. deadlock, unrecoverable panic). The check is lightweight: it confirms the HTTP server loop is responsive and, optionally, that the OIDC JWKS cache is populated. - - *Readiness* (`GET /readyz`) — returns 200 when the pod can accept new work, and 503 when it cannot. The readiness check is tied to the concurrency semaphore: when all permits are in use, the endpoint returns 503, signalling Kubernetes to remove the pod from the Service's endpoint list. New requests are then routed only to pods that still have capacity. Once a permit is released, the probe returns 200 and the pod re-enters the rotation. This provides cluster-level backpressure that is transparent to Trustify — the K8s Service load-balances across ready pods automatically, reducing the likelihood of 429 responses reaching the caller. A 429 is still returned as a last resort if a request arrives between a readiness check and semaphore exhaustion. + - _Liveness_ (`GET /healthz`) — returns 200 if the process is alive. Kubernetes restarts the pod if this probe fails (e.g. deadlock, unrecoverable panic). The check is lightweight: it confirms the HTTP server loop is responsive and, optionally, that the OIDC JWKS cache is populated. + - _Readiness_ (`GET /readyz`) — returns 200 when the pod can accept new work, and 503 when it cannot. The readiness check is tied to the concurrency semaphore: when all permits are in use, the endpoint returns 503, signalling Kubernetes to remove the pod from the Service's endpoint list. New requests are then routed only to pods that still have capacity. Once a permit is released, the probe returns 200 and the pod re-enters the rotation. This provides cluster-level backpressure that is transparent to Trustify — the K8s Service load-balances across ready pods automatically, reducing the likelihood of 429 responses reaching the caller. A 429 is still returned as a last resort if a request arrives between a readiness check and semaphore exhaustion. - **Horizontal Pod Autoscaler (HPA)** — The Wrapper Deployment can be configured with an HPA that scales replicas based on CPU/memory utilization or a custom metric such as the in-flight validation count exposed via a `/metrics` endpoint. Because the readiness probe already removes saturated pods from the Service, the HPA's scaling decisions and the readiness-driven traffic shifting work together: the HPA adds capacity while readiness prevents overload on existing pods. This allows the cluster to absorb demand spikes without requiring a centralized queue. - **Resource limits and requests** — Each Wrapper pod should declare CPU and memory `requests` and `limits` that account for the peak resource usage of the CLI subprocess (the semaphore concurrency multiplied by the per-process footprint). This prevents a burst of validations from starving other workloads on the node. - **NetworkPolicy** — A Kubernetes `NetworkPolicy` can restrict ingress to the Wrapper pods so that only Trustify pods (selected by label) are allowed to reach the `/api/v1/validate` endpoint. This provides a network-layer defense-in-depth on top of the OIDC-based authentication, ensuring that even if a valid token were leaked, it could not be used from outside the trusted namespace. From dd0299173062ad291c8b4a67389a4a6ebe374b7c Mon Sep 17 00:00:00 2001 From: Gilles Dubreuil Date: Mon, 18 May 2026 15:32:46 +0200 Subject: [PATCH 77/77] Use Comforma API instead of CLI --- .../00014-enterprise-contract-integration.md | 513 +++++++++--------- 1 file changed, 253 insertions(+), 260 deletions(-) diff --git a/docs/adrs/00014-enterprise-contract-integration.md b/docs/adrs/00014-enterprise-contract-integration.md index deb35edf1..3d71b4db1 100644 --- a/docs/adrs/00014-enterprise-contract-integration.md +++ b/docs/adrs/00014-enterprise-contract-integration.md @@ -1,16 +1,16 @@ # 00014. Enterprise Contract Integration -Date: 2026-02-03 +Date: 2026-05-15 ## Status -APPROVED +PROPOSED (supersedes previous APPROVED revision dated 2026-02-03) ## Context Trustify provides SBOM storage, analysis, and vulnerability tracking but lacks automated policy enforcement. Organizations need to validate SBOMs against security and compliance policies (licensing, vulnerabilities, provenance) without relying on manual, inconsistent review processes. -Conforma (former Enterprise Contract) is an open-source policy enforcement tool actively maintained by Red Hat. It validates SBOMs against configurable policies and produces structured JSON output. Currently it provides only a CLI; a REST API is planned but with no committed timeline. +Conforma (former Enterprise Contract) is an open-source policy enforcement tool actively maintained by Red Hat. It validates SBOMs against configurable policies and produces structured JSON output. Conforma is developing a REST API; this ADR targets that API as the integration point. ### Requirements @@ -25,42 +25,40 @@ Users need the ability to: ## Decision -We will integrate Conforma into Trustify as a user triggered validation service by interacting with Conforma CLI. -Validation is manually triggered — not automatic on SBOM upload. +Integrate Conforma into Trustify as a user-triggered validation service by communicating directly with the Conforma REST API. +Validation is manually triggered — not automatic on SBOM upload. Trustify stores information to identify (id, name, URL) of Policies. -Conforma CLI is deployed separately from Trustify as either a standalone container or equivalent. +Conforma is assumed to be deployed separately from Trustify as a standalone service. -A Conforma Wrapper (HTTP service) will act as a proxy between Trustify's Policy Verifier service and Conforma CLI. +Trustify's Policy Verifier service communicates directly with the Conforma API over HTTP to submit validation requests and poll for results. -### The Conforma HTTP Wrapper +### Why a separate Conforma service -Policy validation can be be very resource-intensive, especially for large SBOMs with thousands of packages, and it requires a dedicated environment and having the Conforma HTTP Wrapper running alongside the Conforma CLI provides : +Policy validation can be very resource-intensive, especially for large SBOMs with thousands of packages, and running it in a dedicated service provides: -- **Resource isolation** — A long-running or memory-heavy Conforma process cannot degrade Trustify's responsiveness. -- **Independent scaling** — The Conforma Wrapper can be scaled horizontally (more replicas) based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. -- **Failure containment** — An Conforma CLI crash (OOM kill, policy fetch timeout, unexpected CLI error) is isolated to the wrapper which will propagate back to Trustify the failure; This is a terminal state where the user will need to queue a new validation. In case the wrapper crashed, a timeout period will force Trustify to change the state of the queued validations to "Failed". -- **Version independence** — The Conforma Wrapper and Conforma CLI can be upgraded or rolled back on their own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. +- **Resource isolation** — A long-running or memory-heavy validation cannot degrade Trustify's responsiveness. +- **Independent scaling** — The Conforma service can be scaled horizontally based on validation demand without scaling the entire Trustify deployment. Conversely, Trustify can scale for query load without provisioning excess capacity for validation. +- **Failure containment** — A Conforma service failure (OOM kill, policy fetch timeout, internal error) is isolated from Trustify. Trustify detects failures via polling and transitions the validation to a terminal `failed` state. The user can then queue a new validation. +- **Version independence** — The Conforma service can be upgraded or rolled back on its own release cadence, without redeploying Trustify. This is important given Conforma's active development pace. ### Validation process state -Each SBOM + policy pair has two validation states +Each SBOM + policy pair has two validation states. -The validation process state of the - -To track the progress of the external validation process through the Conforma HTTP Wrapper, the following states are used : +To track the progress of the external validation through the Conforma API, the following states are used: - **Queued** — a user has triggered validation; the request is being processed. Other users can see this state, preventing duplicate validation runs for the same SBOM + policy pair. -- **In Progress** — the request has been submitted to Conforma Wrapper. -- **Completed** — the outcome of the request has been received back from Conforma Wrapper. -- **Failed** — an execution error occurred (CLI crash, policy fetch failure, timeout). This is a terminal state; the error is surfaced to the user, who may manually queue a new validation run. +- **In Progress** — the request has been submitted to the Conforma API. +- **Completed** — the outcome has been received from the Conforma API. +- **Failed** — an execution error occurred (API error, policy fetch failure, timeout). This is a terminal state; the error is surfaced to the user, who may manually queue a new validation run. ```mermaid stateDiagram-v2 [*] --> Queued : User triggers validation
INSERT() ON CONFLICT DO NOTHING - Queued --> InProgress : Request submitted to
Conforma Wrapper - InProgress --> Completed : Outcome received - InProgress --> Failed : Execution error
(crash, timeout, fetch failure) + Queued --> InProgress : Request submitted to
Conforma API + InProgress --> Completed : Outcome received
via polling + InProgress --> Failed : Execution error
(API error, timeout, fetch failure) Completed --> [*] : Update validation result
(pass, fail or error) Failed --> [*] ``` @@ -69,33 +67,53 @@ The result of a Policy validation is updated only when the validation process is ### What is stored where -- PostgreSQL: validation process state (`status`), validation outcome (`status`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, status. +- PostgreSQL: validation process state (`status`), validation outcome (`result`), structured results (JSONB), summary statistics, foreign keys to SBOM and policy. Indexed on sbom_id, status. - Storage system: full raw Conforma JSON report, linked from the DB row via `source_document.id`. Keeps DB rows small while preserving audit completeness. -- Not stored: the policy definitions themselves. policy stores references (URLs, OCI refs) that Conforma fetches at runtime. +- Not stored: the policy definitions themselves. Policy stores references (URLs, OCI refs) that Conforma fetches at runtime. Storing full JSON in storage system rather than only a summary was chosen explicitly to preserve audit completeness — callers can always fetch the raw report. The DB results JSONB holds enough structure for filtering and dashboards without duplicating the full payload. ## Consequences -### Conforma HTTP Wrapper +### Direct API Integration + +Trustify communicates directly with the Conforma REST API — no intermediary wrapper or CLI process spawning is involved. The Policy Verifier service in Trustify acts as an HTTP client to the Conforma API. + +The Conforma API contract expected by Trustify is documented in the [Expected Conforma API Contract](#expected-conforma-api-contract) section below; the actual contract will be finalized in collaboration with the Conforma team. + +### Polling-based result retrieval -It's deployed separatly alongside the Conforma CLI instance, monitored, and maintained as a separate component. +Trustify uses a background poller to retrieve validation results from the Conforma API. The poller runs as a recurring task within Trustify's Policy Verifier service: -In Kubernetes or standalone machine deployments, the Conforma Wrapper pod has its own resource requests/limits, independent of the Trustify pod. +1. On each tick (default interval: 15 seconds), it queries for all `policy_validation` rows with `status = 'in_progress'`. +2. For each in-progress validation, it calls the Conforma API status endpoint to check whether the job has completed. +3. If completed, it fetches the full result, parses it, persists the outcome and the raw report, and transitions the row to `completed`. +4. If the Conforma API reports a failure, the row is transitioned to `failed` with the error detail. +5. If the validation has been `in_progress` longer than the configured timeout (default: 5 minutes, overridable per-policy via `timeout_seconds`), the row is transitioned to `failed` with `"Validation timed out"`. -### CLI spawning +This approach is simpler than a callback model because: -Within the Conforma Wrapper, Conforma is invoked via CLI spawning rather than a native API. This introduces an operational dependency (Conforma must be installed and version-pinned on every Conforma Wrapper instance) and per-validation process spawning overhead. These are accepted trade-offs given that no Conforma REST API exists yet. On the Trustify side, the Policy Verifier service interacts with the Conforma Wrapper over HTTP and is built behind an adapter interface, so the implementation can be swapped for a direct Conforma REST client when one becomes available, without changes to the service layer or API. +- Conforma does not need to know Trustify's API or authenticate back to it. +- No bidirectional OIDC configuration is required. +- Trustify controls the polling frequency and timeout behavior entirely. +- Network topology is simpler: only Trustify → Conforma traffic, no reverse path. ### Alternatives Considered -#### In-Process Policy Engine: Rejected +#### CLI + HTTP Wrapper: Superseded -Reimplementing Enterprise Contract logic in Rust would diverge from upstream and create significant maintenance burden. +The previous revision of this ADR (dated 2026-02-03) proposed a Conforma HTTP Wrapper — a custom HTTP service deployed alongside the Conforma CLI that would accept validation requests, spawn `ec validate` as a subprocess, and POST results back to Trustify via a callback endpoint. This approach was approved when no Conforma REST API existed but introduced significant operational complexity: -#### Direct Integration: Rejected +- A custom wrapper service to develop, deploy, monitor, and maintain. +- CLI process spawning with per-validation overhead and OOM/crash risks. +- A bidirectional OIDC configuration (Trustify → Wrapper and Wrapper → Trustify). +- A callback endpoint on Trustify requiring dedicated authentication. -Couple validation integrated within Trustify service through a directly controlled component was simpler but worse for large-scale deployments. +The Conforma REST API eliminates all of these concerns by providing a standard, well-defined integration point that Trustify can consume as an HTTP client. + +#### In-Process Policy Engine: Rejected + +Reimplementing Enterprise Contract logic in Rust would diverge from upstream and create significant maintenance burden. #### Embedded WASM Module: Rejected @@ -103,7 +121,7 @@ Conforma is not available as WASM and would require major upstream changes. #### Batch Processing Queue: Deferred -A Redis/RabbitMQ queue would improve retry handling and priority management; implement if the 429-based rejection approach proves insufficient under real load. +A Redis/RabbitMQ queue would improve retry handling and priority management; implement if the polling-based approach proves insufficient under real load. ## Details @@ -116,11 +134,11 @@ C4Context Person(user, "Trustify User", "Analyst validating SBOMs") System(trustify, "Trustify", "RHTPA") - System_Ext(conforma, "Conforma", "Enterprise Contract policy validation tool") + System_Ext(conforma, "Conforma API", "Enterprise Contract policy validation service") System_Ext(policyRepo, "Policy Repository", "Git repository or storage containing EC policies") Rel(user, trustify, "Request compliance
View compliance status", "API/GUI") - Rel(trustify, conforma, "Triggers policy validation", "HTTP API") + Rel(trustify, conforma, "Submit validation
Poll for results", "HTTP API") Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") UpdateRelStyle(user, trustify, $offsetX="-50", $offsetY="20") @@ -140,47 +158,42 @@ C4Container Container(webui, "Web UI", "Rust/Actix", "Trustify GUI") Container(api, "API Gateway", "Actix-web", "REST API endpoints for SBOM
and compliance operations") ContainerDb(postgres, "PostgreSQL", "DBMS", "Stores SBOM metadata, relationships,
and EC validation results") - Container(policyValidationModule, "Policy Validation Module", "Rust", "Orchestrates validation via
Conforma Wrapper and persists results") + Container(policyValidationModule, "Policy Validation Module", "Rust", "Orchestrates validation via
Conforma API and persists results") ContainerDb(s3, "Object Storage", "S3/Minio", "Stores SBOM documents and EC reports") Container(storage, "Storage Service", "Rust", "Manages document storage
(SBOMs, policy results)") } Container_Boundary(conformaSystem, "Conforma System") { - Container(conformaWrapper, "Conforma Wrapper", "Rust/Actix", "HTTP Wrapper") - System_Ext(conforma, "Conforma CLI", "External policy validation tool") + Container(conformaApi, "Conforma API", "REST Service", "Policy validation service") } Container_Boundary(policySystem, "Policy System") { System_Ext(policyRepo, "Policy Repository", "Git repository with EC policies") } - Container_Boundary(oidc, "OIDC") { - System_Ext(oidc, "OIDC", "OPenID Connect OAuth 2.0 - ", "") + Container_Boundary(oidcBoundary, "OIDC") { + System_Ext(oidc, "OIDC", "OpenID Connect OAuth 2.0") } Rel(user, webui, "Views compliance status", "HTTP API") Rel(user, api, "Views compliance status", "HTTP API") Rel(webui, api, "API calls", "JSON/HTTP API") Rel(api, policyValidationModule, "Triggers validation", "Function call") - Rel(policyValidationModule, conformaWrapper, "POST /api/v1/validate", "HTTP API formData") - Rel(conformaWrapper, api, "POST /api/v2/policy/{id}/validation/{validation_id}/result", "HTTP API") - Rel(conformaWrapper, conforma, "ec validate input {SBOM} {policy}", "Spawned command") + Rel(policyValidationModule, conformaApi, "POST /validate
GET /validate/{id}", "HTTP API") Rel(policyValidationModule, postgres, "Saves validation
results", "SQL") Rel(policyValidationModule, storage, "Stores EC reports", "Function call") Rel(storage, s3, "Persists reports", "S3 API") - Rel(conforma, policyRepo, "Fetches policies", "Git/HTTPS") - Rel(conformaWrapper, oidc, "Authenticate", "OAuth API") + Rel(conformaApi, policyRepo, "Fetches policies", "Git/HTTPS") + Rel(policyValidationModule, oidc, "Obtain token", "OAuth API") UpdateRelStyle(user, webui, $offsetX="-60", $offsetY="30") UpdateRelStyle(user, api, $offsetX="-60", $offsetY="-50") UpdateRelStyle(webui, api, $offsetX="-40", $offsetY="10") - UpdateRelStyle(policyValidationModule, conformaWrapper, $offsetX="-50", $offsetY="-20") - UpdateRelStyle(conformaWrapper, api, $offsetX="-60", $offsetY="-10") + UpdateRelStyle(policyValidationModule, conformaApi, $offsetX="-50", $offsetY="-20") UpdateRelStyle(policyValidationModule, postgres, $offsetX="-40", $offsetY="10") UpdateRelStyle(storage, s3, $offsetX="-40", $offsetY="10") - UpdateRelStyle(conforma, policyRepo, $offsetX="-40", $offsetY="100") - UpdateRelStyle(conformaWrapper, oidc, $offsetX="30", $offsetY="40") + UpdateRelStyle(conformaApi, policyRepo, $offsetX="-40", $offsetY="100") + UpdateRelStyle(policyValidationModule, oidc, $offsetX="30", $offsetY="40") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` @@ -195,46 +208,43 @@ C4Component Container(api, "API Gateway", "Actix-web", "REST API for EC operations") Container_Boundary(policyValidationModule, "Policy Validation Module") { Container(policyEndpoints, "Policy Endpoints", "Actix-web handlers", "REST endpoints for validation operations") - Component(policyVeriferService, "Policy Verifier service", "Business logic", "Orchestrates validation workflow") + Component(policyVerifierService, "Policy Verifier service", "Business logic", "Orchestrates validation workflow") Component(resultParser, "Result Parser", "JSON parser", "Parses Conforma output into structured data") Component(policyManager, "Policy Manager", "Business logic", "Manages EC policy references and
configuration") Component(resultPersistence, "Result Persistence", "Database layer", "Saves validation results") + Component(validationPoller, "Validation Poller", "Background task", "Polls Conforma API for
in-progress validation results") + Component(conformaClient, "Conforma Client", "HTTP client", "Communicates with
Conforma REST API") } } Deployment_Node(external, "External System") { - Deployment_Node(trustifyPod, "Trustify Pod") { - Component(conformaWrapper, "Conforma Wrapper", "Actix-web handlers", "HTTP API") - } - Deployment_Node(conformaPod, "Conforma Pod") { - System_Ext(conforma, "Conforma CLI", "Enterprise Contract validation tool") - } + System_Ext(conformaApi, "Conforma API", "Enterprise Contract validation service") } Container_Boundary(dbms, "Database") { ContainerDb(postgres, "PostgreSQL", "Database", "Stores validation results and policy references") } + Container_Boundary(storage, "S3 System") { System_Ext(s3, "S3 Object Storage", "Stores SBOM documents and reports") } - Rel(api, policyEndpoints, "POST /api/v2/policy/{id}/validation,\nGET /api/v2/policy/{id}/validation/report,\nGET /api/v2/policy/{id}/validation/report/history,\nGET /api/v2/policy/{id}/validation/report/{result_id},\nPOST /api/v2/policy/{id}/validation/{validation_id}/result", "JSON/HTTPS") - Rel(policyEndpoints, policyVeriferService, "validate_sbom() / get_ec_report()", "Function call") - Rel(policyVeriferService, policyManager, "get_policy_config()", "Function call") + Rel(api, policyEndpoints, "POST /api/v2/policy/{id}/validation,\nGET /api/v2/policy/{id}/validation/report,\nGET /api/v2/policy/{id}/validation/report/history,\nGET /api/v2/policy/{id}/validation/report/{result_id}", "JSON/HTTPS") + Rel(policyEndpoints, policyVerifierService, "validate_sbom() / get_ec_report()", "Function call") + Rel(policyVerifierService, policyManager, "get_policy_config()", "Function call") Rel(policyManager, postgres, "SELECT policy", "SQL") - Rel(policyVeriferService, conformaWrapper, "POST /api/v1/validate → returns {id}", "HTTP") - Rel(conformaWrapper, conforma, "ec validate", "Process spawn") - Rel(conformaWrapper, api, "POST /api/v2/policy/{id}/validation/{validation_id}/result", "JSON/HTTPS") - Rel(policyVeriferService, resultParser, "parse_output()", "Function call") - Rel(policyVeriferService, resultPersistence, "save_results()", "Function call") - Rel(resultPersistence, postgres, "INSERT policy_validation", "SQL") - Rel(policyVeriferService, s3, "Store EC report", "S3 API") + Rel(policyVerifierService, conformaClient, "submit_validation()", "Function call") + Rel(conformaClient, conformaApi, "POST /validate\nGET /validate/{id}", "HTTP") + Rel(validationPoller, conformaClient, "check_status()", "Function call") + Rel(validationPoller, resultParser, "parse_output()", "Function call") + Rel(validationPoller, resultPersistence, "save_results()", "Function call") + Rel(resultPersistence, postgres, "INSERT/UPDATE policy_validation", "SQL") + Rel(policyVerifierService, s3, "Store EC report", "S3 API") UpdateRelStyle(api, policyEndpoints, $offsetX="-50", $offsetY="-50") - UpdateRelStyle(policyEndpoints, policyVeriferService, $offsetX="-60", $offsetY="+40") - UpdateRelStyle(policyVeriferService, conformaWrapper, $offsetX="-20", $offsetY="10") - UpdateRelStyle(conformaWrapper, api, $offsetX="20", $offsetY="-40") - UpdateRelStyle(policyVeriferService, resultParser, $offsetX="-60", $offsetY="+0") - UpdateRelStyle(policyVeriferService, resultPersistence, $offsetX="-60", $offsetY="+80") + UpdateRelStyle(policyEndpoints, policyVerifierService, $offsetX="-60", $offsetY="+40") + UpdateRelStyle(policyVerifierService, conformaClient, $offsetX="-20", $offsetY="10") + UpdateRelStyle(policyVerifierService, resultParser, $offsetX="-60", $offsetY="+0") + UpdateRelStyle(policyVerifierService, resultPersistence, $offsetX="-60", $offsetY="+80") UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2") ``` @@ -251,7 +261,8 @@ sequenceDiagram participant PM as Policy Manager participant DB as PostgreSQL participant S3 as Object Storage - participant Wrapper as Conforma Wrapper (HTTP) + participant CC as Conforma Client + participant CA as Conforma API User->>API: POST /api/v2/policy//validation?sbom_id= API->>EP: dispatch request @@ -282,58 +293,67 @@ sequenceDiagram VS->>S3: retrieve_sbom_document(sbom_id) S3-->>VS: SBOM document (JSON/XML) - VS->>Wrapper: POST /api/v1/validate {SBOM, policy_ref} - Wrapper-->>VS: 202 Accepted {validation_id} + VS->>CC: submit_validation(sbom, policy_config) + CC->>CA: POST /validate {sbom, policy_ref, policy_paths, ...} + CA-->>CC: 202 Accepted {job_id} + CC-->>VS: job_id - VS->>DB: INSERT policy_validation (status='in_progress', status='pending', sbom_id, policy_id, validation_id, ...) + VS->>DB: INSERT policy_validation (status='in_progress', result='pending', sbom_id, policy_id, conforma_job_id, ...) VS-->>EP: 202 Accepted {validation_id} EP-->>API: 202 Accepted {validation_id} API-->>User: 202 Accepted {validation_id} ``` -### Sequence Diagram — Async Validation Processing +### Sequence Diagram — Async Result Polling ```mermaid sequenceDiagram autonumber - participant API as Trustify API - participant EP as Policy Endpoints - participant VS as Policy Verifier service + participant Poller as Validation Poller + participant CC as Conforma Client + participant CA as Conforma API + participant RP as Result Parser participant DB as PostgreSQL participant S3 as Object Storage - participant Wrapper as Conforma Wrapper (HTTP) - participant Conf as Conforma CLI - participant OIDC as OIDC Provider - - Note over Wrapper: Wrapper holds validation_id from the initial request - - Wrapper->>Wrapper: Write SBOM to temp file - Wrapper->>Conf: ec validate input --file "$SBOM_PATH" --policy --output json --show-successes --info - alt Exit code 0 — pass - Conf-->>Wrapper: {result: "PASS", violations: []} - else Exit code 1 — fail (policy violations) - Conf-->>Wrapper: {result: "FAIL", violations: [{rule, severity, message}]} - else Exit code 2+ — execution error - Conf-->>Wrapper: stderr: "Policy file not found" - end - - Wrapper->>Wrapper: Cleanup temp files - Wrapper->>API: POST /api/v2/policy/{id}/validation/{validation_id}/result {conforma JSON output}
Authorization: Bearer - API->>EP: dispatch callback - EP->>VS: process_validation_result(validation_id, result) - - alt Pass - VS->>DB: UPDATE policy_validation SET status='completed', result='pass', results=[] - VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET source_document.id = ? - else Fail - VS->>DB: UPDATE policy_validation SET status='completed', result='fail', results=json - VS->>S3: store_validation_report(result_id, full_json) - VS->>DB: UPDATE SET source_document.id = ? - else Error - VS->>DB: UPDATE policy_validation SET status='completed', result='error', error_message=detail - Note over VS,DB: Failed is terminal. User may manually queue a new validation (new row). + Note over Poller: Background task runs every 15s + + Poller->>DB: SELECT * FROM policy_validation WHERE status = 'in_progress' + DB-->>Poller: [validation_1, validation_2, ...] + + loop For each in-progress validation + Poller->>CC: check_status(conforma_job_id) + CC->>CA: GET /validate/{job_id} + + alt Job completed — pass + CA-->>CC: {status: "completed", result: {success: true, ...}} + CC-->>Poller: ConformaResult + Poller->>RP: parse_output(conforma_result) + RP-->>Poller: ParsedResult {result, results[], summary} + Poller->>DB: UPDATE policy_validation SET status='completed', result='pass', results=json + Poller->>S3: store_validation_report(validation_id, full_json) + Poller->>DB: UPDATE SET source_document_id = ? + else Job completed — fail (policy violations) + CA-->>CC: {status: "completed", result: {success: false, violations: [...]}} + CC-->>Poller: ConformaResult + Poller->>RP: parse_output(conforma_result) + RP-->>Poller: ParsedResult {result, results[], summary} + Poller->>DB: UPDATE policy_validation SET status='completed', result='fail', results=json + Poller->>S3: store_validation_report(validation_id, full_json) + Poller->>DB: UPDATE SET source_document_id = ? + else Job failed — execution error + CA-->>CC: {status: "failed", error: "Policy file not found"} + CC-->>Poller: Error + Poller->>DB: UPDATE policy_validation SET status='failed', result='error', error_message=detail + else Job still running + CA-->>CC: {status: "running"} + Note over Poller: Check timeout + alt Exceeded timeout + Poller->>DB: UPDATE policy_validation SET status='failed', error_message='Validation timed out' + else Within timeout + Note over Poller: Skip, will check again next tick + end + end end ``` @@ -356,13 +376,12 @@ sequenceDiagram | `auth` | object | no | Credentials for private repos; sensitive values encrypted via `ring::aead` AES-256-GCM (never logged) | | `auth.type` | string | yes (if `auth`) | `"token"`, `"ssh_key"`, or `"none"` | | `auth.token_encrypted` | string | no | AES-256-GCM encrypted bearer/PAT token, prefixed with encryption scheme | -| `policy_paths` | string[] | no | Sub-paths within the repo to evaluate (maps to Conforma `--policy` source paths) | +| `policy_paths` | string[] | no | Sub-paths within the repo to evaluate | | `exclude` | string[] | no | Rule codes to skip during validation | | `include` | string[] | no | If non-empty, only these rule codes are evaluated | | `timeout_seconds` | integer | no | Per-policy override of the default 5-minute execution timeout | -| `extra_args` | string[] | no | Additional CLI flags forwarded verbatim to Conforma | -`policy.configuration` example : +`policy.configuration` example: ```json { @@ -374,8 +393,7 @@ sequenceDiagram "policy_paths": ["policy/lib", "policy/release"], "exclude": ["hello_world.minimal_packages"], "include": [], - "timeout_seconds": 300, - "extra_args": ["--strict"] + "timeout_seconds": 300 } ``` @@ -384,6 +402,7 @@ sequenceDiagram - `id` (UUID, PK) - `sbom_id` (UUID, FK → sbom) - `policy_id` (UUID, FK → policy) +- `conforma_job_id` (VARCHAR) - Job ID returned by the Conforma API, used for polling - `status` (ENUM) - 'null', 'queued', 'in_progress', 'completed', 'failed' - `error`(TEXT) - Error message - `result` (ENUM) - 'null', 'fail', 'pass' or 'error' @@ -400,15 +419,15 @@ sequenceDiagram **`policy_validation.results` JSONB model:** -| Field | Type | Required | Description | -| ---------------------- | ------ | -------- | ------------------------------------------------------- | -| `severity` | string | yes | `"violation"`, `"warning"`, or `"success"` | -| `msg` | string | yes | Human-readable message describing the check outcome | -| `metadata` | object | yes | Rule metadata, preserved as-is from Conforma CLI output | -| `metadata.code` | string | yes | Rule identifier for filtering and deduplication | -| `metadata.title` | string | yes | Short rule title | -| `metadata.description` | string | no | Detailed explanation of what the rule checks | -| `metadata.solution` | string | no | Suggested remediation (absent for successes) | +| Field | Type | Required | Description | +| ---------------------- | ------ | -------- | --------------------------------------------------- | +| `severity` | string | yes | `"violation"`, `"warning"`, or `"success"` | +| `msg` | string | yes | Human-readable message describing the check outcome | +| `metadata` | object | yes | Rule metadata, preserved as-is from Conforma output | +| `metadata.code` | string | yes | Rule identifier for filtering and deduplication | +| `metadata.title` | string | yes | Short rule title | +| `metadata.description` | string | no | Detailed explanation of what the rule checks | +| `metadata.solution` | string | no | Suggested remediation (absent for successes) | `policy_validation.results` example: @@ -450,7 +469,7 @@ sequenceDiagram | Field | Type | Required | Description | | ------------------ | ------ | -------- | ------------------------------------------ | -| `conforma_version` | string | yes | Version of Conforma CLI (e.g. `"v0.8.83"`) | +| `conforma_version` | string | yes | Version of Conforma API (e.g. `"v0.8.83"`) | `policy_validation.type_metadata` example: @@ -523,8 +542,6 @@ struct PolicyConfiguration { include: Vec, #[serde(default, skip_serializing_if = "Option::is_none")] timeout_seconds: Option, - #[serde(default, skip_serializing_if = "Vec::is_empty")] - extra_args: Vec, } ``` @@ -543,6 +560,9 @@ struct PolicyValidation { id: String, sbom_id: String, policy_id: String, + /// Job ID from the Conforma API, used for polling + #[serde(default, skip_serializing_if = "Option::is_none")] + conforma_job_id: Option, /// Lifecycle: `'null'`, `'queued'`, `'in_progress'`, `'completed'`, `'failed'` status: String, #[serde(default, skip_serializing_if = "Option::is_none")] @@ -622,7 +642,6 @@ POST /api/v2/policy/{id}/validation # Trigger GET /api/v2/policy/{id}/validation/report # Get latest validation result GET /api/v2/policy/{id}/validation/report/history # Get validation history GET /api/v2/policy/{id}/validation/report/{result_id} # Download full report -POST /api/v2/policy/{id}/validation/{validation_id}/result # Callback for validation result ``` ### Permissions @@ -633,7 +652,7 @@ The policy module introduces the following permissions, following the existing T | --------------- | ------------------------------------------------------------------ | | `create.policy` | Create policy references and trigger validations | | `read.policy` | List/get policy references and read validation results and reports | -| `update.policy` | Update policy references and post validation results (callback) | +| `update.policy` | Update policy references | | `delete.policy` | Delete policy references | These permissions map to the default OIDC scope groups: @@ -647,20 +666,17 @@ These permissions map to the default OIDC scope groups: Endpoint permission requirements: -| Endpoint | Permission | -| ------------------------------------------------------------ | --------------- | -| `POST /api/v2/policy` | `create.policy` | -| `GET /api/v2/policy` | `read.policy` | -| `GET /api/v2/policy/{id}` | `read.policy` | -| `PUT /api/v2/policy/{id}` | `update.policy` | -| `DELETE /api/v2/policy/{id}` | `delete.policy` | -| `POST /api/v2/policy/{id}/validation` | `create.policy` | -| `GET /api/v2/policy/{id}/validation/report` | `read.policy` | -| `GET /api/v2/policy/{id}/validation/report/history` | `read.policy` | -| `GET /api/v2/policy/{id}/validation/report/{result_id}` | `read.policy` | -| `POST /api/v2/policy/{id}/validation/{validation_id}/result` | `update.policy` | - -The Conforma Wrapper's OIDC client (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration)) should be registered with the minimal scope required — only `update:document` (granting `update.policy`) — so it can post results to the callback endpoint. +| Endpoint | Permission | +| ------------------------------------------------------- | --------------- | +| `POST /api/v2/policy` | `create.policy` | +| `GET /api/v2/policy` | `read.policy` | +| `GET /api/v2/policy/{id}` | `read.policy` | +| `PUT /api/v2/policy/{id}` | `update.policy` | +| `DELETE /api/v2/policy/{id}` | `delete.policy` | +| `POST /api/v2/policy/{id}/validation` | `create.policy` | +| `GET /api/v2/policy/{id}/validation/report` | `read.policy` | +| `GET /api/v2/policy/{id}/validation/report/history` | `read.policy` | +| `GET /api/v2/policy/{id}/validation/report/{result_id}` | `read.policy` | ### POST `/api/v2/policy` @@ -793,7 +809,7 @@ Deleting a policy will fail if there are validation results referencing it. ### POST `/api/v2/policy/{id}/validation` -Trigger policy validation for a given SBOM. The validation is performed asynchronously by the Conforma Wrapper; a `validation_id` is returned immediately. +Trigger policy validation for a given SBOM. The validation is performed asynchronously via the Conforma API; a `validation_id` is returned immediately. If a validation is already in progress for the same SBOM + policy pair, the request is rejected with 409 Conflict. @@ -819,7 +835,8 @@ If a validation is already in progress for the same SBOM + policy pair, the requ - 403 - if the user was authenticated but not authorized - 404 - if the SBOM or policy was not found - 409 - if a validation is already in progress for this SBOM + policy pair -- 429 - if the Conforma Wrapper has reached its concurrency limit +- 502 - if the Conforma API is unreachable or returned an unexpected error +- 503 - if the Conforma API reported it cannot accept work at this time ### GET `/api/v2/policy/{id}/validation/report` @@ -900,65 +917,67 @@ Download the full raw Conforma JSON report from storage. - 403 - if the user was authenticated but not authorized - 404 - if the validation result or report was not found -### POST `/api/v2/policy/{id}/validation/{validation_id}/result` +## Expected Conforma API Contract + +The following API contract is what Trustify expects from the Conforma REST API. This contract is aspirational and subject to change once the Conforma API is finalized. Trustify's Conforma client adapter is built behind a trait interface so the implementation can be adjusted to match the actual API without changes to the service layer. -Callback endpoint used by the Conforma Wrapper to post the validation result back to Trustify after Conforma CLI execution completes. +### POST `/validate` -This endpoint is not intended for end-user use. Because Trustify enforces OAuth 2.0 authentication on all API endpoints, the Conforma Wrapper must present a valid Bearer token obtained via the **OAuth 2.0 Client Credentials Grant** (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration) below). +Submit an SBOM document and policy reference for validation. #### Request -| part | name | type | description | -| ------ | --------------- | -------- | -------------------------------------------------------------------------------------- | -| path | `id` | `String` | Policy id; must match the policy under which the validation was started | -| path | `validation_id` | `String` | ID of the validation (returned in the 202 response) | -| header | `Authorization` | `String` | `Bearer ` — token obtained from the OIDC provider via Client Credentials | -| body | - | raw JSON | The raw Conforma CLI JSON output | +| part | name | type | description | +| --------- | --------------- | -------- | -------------------------------------------------------------------- | +| header | `Authorization` | `String` | `Bearer ` — token for Conforma API authentication | +| multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | +| multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | +| multipart | `policy_paths` | `String` | JSON-encoded array of sub-paths within the repo (optional) | +| multipart | `exclude` | `String` | JSON-encoded array of rule codes to skip (optional) | +| multipart | `include` | `String` | JSON-encoded array of rule codes to evaluate exclusively (optional) | #### Response -- 204 - the result was accepted and persisted -- 400 - if the request could not be understood or the JSON is malformed -- 401 - if the caller was not authenticated (missing or invalid Bearer token) -- 403 - if the caller was not authorized (token lacks required scope/role) -- 404 - if the policy or validation ID was not found, or the validation does not belong to this policy -- 409 - if the validation already has a result (duplicate callback) - -## Conforma Wrapper API Endpoints +- 202 - the validation was accepted and will be processed asynchronously -### POST `/api/v1/validate` + ```json + { + "job_id": "" + } + ``` -Accept an SBOM document and policy reference, spawn a Conforma CLI validation, and asynchronously post the result back to the Trustify callback endpoint. +- 400 - if the request could not be understood or required fields are missing +- 401 - if the caller was not authenticated +- 503 - if the service cannot accept work at this time -Because the Conforma Wrapper is a network-accessible service, it **must authenticate incoming requests** using the same OIDC provider that Trustify uses. Callers must present a valid Bearer token obtained from the shared OIDC provider. The Wrapper validates the token by fetching the provider's JWKS and verifying the signature, expiry, and issuer — exactly the same mechanism Trustify uses for its own endpoints (see `trustify-auth` / `Authenticator`). This ensures that only authorized Trustify instances (or other permitted clients) can trigger validations. +### GET `/validate/{job_id}` -The Conforma Wrapper must also be pre-configured with OIDC client credentials so it can authenticate to Trustify when posting results back (see [Conforma Wrapper OIDC Configuration](#conforma-wrapper-oidc-configuration)). +Check the status of a previously submitted validation job and retrieve results when complete. #### Request -| part | name | type | description | -| --------- | --------------- | -------- | -------------------------------------------------------------------------------------- | -| header | `Authorization` | `String` | `Bearer ` — token obtained from the shared OIDC provider | -| multipart | `sbom` | file | The SBOM document to validate (JSON or XML) | -| multipart | `policy_ref` | `String` | Policy source URL (e.g. `git://github.com/org/policy-repo?ref=main`) | -| multipart | `callback_url` | `String` | Trustify callback URL (`/api/v2/policy/{policy_id}/validation/{validation_id}/result`) | -| multipart | `extra_args` | `String` | Additional CLI flags forwarded to Conforma (optional, JSON-encoded array) | +| part | name | type | description | +| ------ | --------------- | -------- | --------------------------------------------------------------- | +| path | `job_id` | `String` | Job ID returned in the 202 response | +| header | `Authorization` | `String` | `Bearer ` — token for Conforma API authentication | #### Response -- 202 - the validation was accepted and will be processed asynchronously +- 200 - job status and result (if completed) - ```rust - #[derive(Serialize, Deserialize)] - struct WrapperValidationAccepted { - validation_id: Uuid, + ```json + { + "job_id": "", + "status": "pending | running | completed | failed", + "result": { ... }, + "error": "..." } ``` -- 400 - if the request could not be understood or required fields are missing -- 401 - if the caller was not authenticated (missing or invalid Bearer token) -- 403 - if the caller was authenticated but not authorized (token lacks required scope/role) -- 429 - if the concurrency semaphore is exhausted (too many concurrent validations) + When `status` is `"completed"`, `result` contains the full Conforma validation output (same structure as the current Conforma CLI JSON output). When `status` is `"failed"`, `error` contains a description of the failure. + +- 401 - if the caller was not authenticated +- 404 - if the job ID was not found ## File Structure @@ -978,113 +997,87 @@ modules/policy/ │ │ ├── mod.rs │ │ ├── ec_service.rs # Main orchestration │ │ ├── policy_manager.rs # Policy configuration -│ │ └── result_parser.rs # Output parsing +│ │ ├── result_parser.rs # Output parsing +│ │ └── validation_poller.rs # Background polling task │ └── client/ -│ └── conforma.rs # Conforma client adapter -modules/conforma_wrapper -├── build.rs -├── Cargo.toml -└── src/ - ├── endpoints/ - │ └── mod.rs # REST endpoints - └── lib.rs +│ └── conforma.rs # Conforma API HTTP client ``` ## Technical Considerations -#### Conforma Wrapper OIDC Configuration - -The Conforma Wrapper has two distinct OIDC responsibilities that share the **same OIDC provider** (e.g. the Keycloak realm used by Trustify): - -1. **Inbound authentication** — validate Bearer tokens on incoming requests to `/api/v1/validate`, ensuring only authorized callers (Trustify, CI systems, etc.) can trigger validations. -2. **Outbound authentication** — obtain an access token via the Client Credentials Grant to authenticate when posting results back to the Trustify callback endpoint. +#### Conforma API Client Configuration -Because both directions use the same OIDC provider, the Wrapper only needs a single issuer URL. It fetches the provider's discovery document (`.well-known/openid-configuration`) once at startup to obtain the JWKS URI (for inbound token validation) and the token endpoint (for outbound token acquisition). - -Conversely, the **Conforma client adapter** in Trustify (`modules/policy/src/client/conforma.rs`) also acts as a server: it exposes the callback endpoint (`POST /api/v2/policy/{policy_id}/validation/{validation_id}/result`) that receives validation results from the Conforma Wrapper. This endpoint must be protected so that only the Wrapper — authenticated via its Client Credentials token — can post results. Because the callback endpoint is a standard Trustify REST endpoint, it is protected by Trustify's existing OIDC authentication middleware (`trustify-auth` / `Authenticator`), which validates the Bearer token the Wrapper obtained during its outbound token acquisition. The OIDC client registered for the Wrapper must be granted the minimum role or scope required to call this callback (e.g. `policy:write`), and Trustify must reject tokens that lack the necessary permission with 403 Forbidden. +Trustify's Conforma client requires the following configuration to communicate with the Conforma API: ##### Environment Variables -The following environment variables (or equivalent configuration) must be set at deployment time: - -| Variable | Required | Description | -| ----------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -| `TRUSTIFY_OIDC_ISSUER_URL` | yes | OIDC issuer URL (e.g. `https://keycloak.example.com/realms/trustify`). Used for discovery of JWKS, token endpoint, and issuer validation. | -| `TRUSTIFY_OIDC_CLIENT_ID` | yes | Client ID registered in the OIDC provider for the Conforma Wrapper (used for both inbound audience validation and outbound token grant) | -| `TRUSTIFY_OIDC_CLIENT_SECRET` | yes | Client secret for the registered client (used for the outbound Client Credentials Grant) | -| `TRUSTIFY_OIDC_SCOPE` | no | OAuth scope to request on outbound token grants (defaults to provider default; set if Trustify requires a specific scope) | -| `TRUSTIFY_OIDC_TLS_INSECURE` | no | Disable TLS certificate validation for the OIDC provider (default `false`; only for development) | -| `TRUSTIFY_OIDC_TLS_CA_CERTIFICATES` | no | Additional CA certificates to trust when communicating with the OIDC provider | - -##### Inbound Token Validation +| Variable | Required | Description | +| ---------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------- | +| `CONFORMA_API_URL` | yes | Base URL of the Conforma API (e.g. `https://conforma.example.com`) | +| `CONFORMA_API_AUTH_TYPE` | no | Authentication type: `"oidc"` (default) or `"api_key"` | +| `CONFORMA_OIDC_ISSUER_URL` | cond. | OIDC issuer URL, required when `auth_type` is `"oidc"` | +| `CONFORMA_OIDC_CLIENT_ID` | cond. | OIDC client ID for obtaining tokens, required when `auth_type` is `"oidc"` | +| `CONFORMA_OIDC_CLIENT_SECRET` | cond. | OIDC client secret, required when `auth_type` is `"oidc"` | +| `CONFORMA_API_KEY` | cond. | API key, required when `auth_type` is `"api_key"` | +| `CONFORMA_TLS_INSECURE` | no | Disable TLS certificate validation (default `false`; only for development) | +| `CONFORMA_TLS_CA_CERTIFICATES` | no | Additional CA certificates to trust when communicating with the Conforma API | +| `CONFORMA_POLL_INTERVAL_SECONDS` | no | Polling interval for the validation poller (default: 15 seconds) | +| `CONFORMA_DEFAULT_TIMEOUT_SECONDS` | no | Default validation timeout (default: 300 seconds), overridable per-policy via `configuration.timeout_seconds` | -On startup the Wrapper fetches the JWKS from the OIDC provider's discovery endpoint and caches it. For each incoming request to `/api/v1/validate`: +##### Authentication -1. Extract the `Authorization: Bearer ` header. -2. Verify the JWT signature against the cached JWKS (refresh on cache miss for key rotation). -3. Validate standard claims: `iss` matches `TRUSTIFY_OIDC_ISSUER_URL`, token is not expired, `aud` or `azp` includes the expected client. -4. On failure return 401 Unauthorized. +When `auth_type` is `"oidc"`, the Conforma client uses the OAuth 2.0 Client Credentials Grant to obtain an access token from the OIDC provider. The token is cached and refreshed proactively before expiry (with a 30-second safety margin). -This follows the same validation logic that Trustify's own `Authenticator` uses (see `trustify-auth` crate), so the Wrapper can reuse or mirror that implementation. +When `auth_type` is `"api_key"`, the client sends the key in the `Authorization` header as `Bearer `. -##### Outbound Token Acquisition - -For posting results back to Trustify the Wrapper uses the **OAuth 2.0 Client Credentials Grant**: - -**Token lifecycle**: The Wrapper should cache the access token and refresh it proactively before expiry (using the `expires_in` value from the token response). This avoids a token request on every callback and handles clock skew by refreshing with a safety margin (e.g. 30 seconds before expiry). - -**Failure handling**: If the token request fails (OIDC provider unreachable, invalid credentials), the Wrapper cannot deliver the callback. The validation remains in `in_progress` state until Trustify's timeout mechanism marks it as `failed` (see below). The Wrapper should log the token acquisition error and may retry with exponential backoff before giving up. - -**Trustify timeout mechanism**: A background reaper task runs periodically inside the Policy Verifier service (default interval: 60 seconds). On each tick it queries for `policy_validation` rows whose `status` is `in_progress` and whose `updated_at` timestamp is older than the configured timeout (default: the policy's `timeout_seconds`, falling back to the global default of 5 minutes). Each stale row is transitioned to `status = 'failed'` with an `error` of `"Validation timed out"`. This covers every failure scenario where the Wrapper never delivers a callback — CLI hang, Wrapper crash, network partition, or token acquisition failure. The reaper uses an `UPDATE … WHERE status = 'in_progress' AND updated_at < NOW() - interval` query so it is idempotent and safe to run from multiple Trustify replicas concurrently. +The authentication mechanism is defined behind a trait so additional schemes can be added without modifying the core client logic. ##### Security Considerations -- The client secret must be stored securely (e.g. Kubernetes Secret, vault injection) and never logged. -- The OIDC client registered for the Conforma Wrapper should be a **dedicated confidential client** (e.g. `conforma-wrapper`) in the OIDC provider, separate from clients used by human users or other services. This client should be granted a narrow, purpose-specific scope — `trustify:policy-callback` — that maps to the single permission needed to post validation results to the callback endpoint. Trustify's authorization middleware must enforce this scope on the callback route (`POST /api/v2/policy/{policy_id}/validation/{validation_id}/result`), rejecting tokens that lack it with 403 Forbidden. By using a dedicated client and scope rather than a broad role like `policy:write`, the blast radius of a compromised credential is limited to callback delivery only — the Wrapper cannot read, delete, or otherwise mutate policies or other Trustify resources. -- TLS is required for all OIDC communication (provider discovery, JWKS fetch, token endpoint). -- The Wrapper should reject tokens from unexpected issuers; the `TRUSTIFY_OIDC_ISSUER_URL` acts as an allowlist of exactly one trusted issuer. - -#### Conforma CLI Execution - -The Conforma Wrapper invokes Conforma CLI via process spawning (e.g., `tokio::process::Command`). All arguments are passed as an array — never as a shell string — to prevent CLI injection. Execution has a configurable timeout (default 5 minutes); SBOMs are written to a temp file and passed by path in order to avoid OOM issues as SBOM can be very large file they shouldn't not be transfered via STDIN stream. - -Exit codes are treated as follows: 0 = pass, 1 = policy violations (expected failure, not an error), 2+ = execution error. It is important to distinguish 1 from 2+ in error handling — a policy violation is a valid result that should be surfaced to the user, not treated as a system failure. +- Client secrets and API keys must be stored securely (e.g. Kubernetes Secret, vault injection) and never logged. +- TLS is required for all communication with the Conforma API in production. +- The OIDC client registered for Trustify's Conforma integration should use a narrow scope that grants only the ability to submit and query validations. #### Concurrency and Backpressure Concurrency is controlled at two levels: -- **Trustify (duplicate prevention)** — Before forwarding a request to the Conforma Wrapper, the Policy Verifier service checks whether a validation is already queued or in progress for the same SBOM + policy pair. If one exists, the request is rejected with 409 Conflict, preventing duplicate work. -- **Conforma Wrapper (resource protection)** — Concurrent Conforma CLI processes are bounded by a semaphore (default: 5). When the semaphore is exhausted, the Wrapper returns 429 Too Many Requests, which Trustify propagates to the caller. This makes the capacity limit explicit so that callers (e.g., CI pipelines) can implement their own retry with backoff. +- **Trustify (duplicate prevention)** — Before forwarding a request to the Conforma API, the Policy Verifier service checks whether a validation is already queued or in progress for the same SBOM + policy pair. If one exists, the request is rejected with 409 Conflict, preventing duplicate work. +- **Conforma API (resource protection)** — The Conforma API manages its own internal concurrency limits. When it cannot accept additional work, it returns an appropriate error (e.g. 503 Service Unavailable), which Trustify propagates to the caller. This delegates capacity management to the service that owns the resources, avoiding the need for Trustify to maintain a semaphore or concurrency limit for an external service. -If demand grows beyond what the semaphore-based approach can handle, a proper queue (Redis/RabbitMQ) is a deferred alternative (see _Batch Processing Queue_ in Alternatives Considered above). +#### Validation Poller -##### Kubernetes Deployment +The validation poller is a background task within the Policy Verifier service that periodically checks the Conforma API for completed validations: -When both Trustify and the Conforma Wrapper are deployed on a Kubernetes cluster, native K8s primitives can complement the application-level concurrency controls described above: - -- **Readiness and liveness probes** — The Conforma Wrapper exposes two health endpoints that Kubernetes uses to manage pod lifecycle and traffic routing: - - _Liveness_ (`GET /healthz`) — returns 200 if the process is alive. Kubernetes restarts the pod if this probe fails (e.g. deadlock, unrecoverable panic). The check is lightweight: it confirms the HTTP server loop is responsive and, optionally, that the OIDC JWKS cache is populated. - - _Readiness_ (`GET /readyz`) — returns 200 when the pod can accept new work, and 503 when it cannot. The readiness check is tied to the concurrency semaphore: when all permits are in use, the endpoint returns 503, signalling Kubernetes to remove the pod from the Service's endpoint list. New requests are then routed only to pods that still have capacity. Once a permit is released, the probe returns 200 and the pod re-enters the rotation. This provides cluster-level backpressure that is transparent to Trustify — the K8s Service load-balances across ready pods automatically, reducing the likelihood of 429 responses reaching the caller. A 429 is still returned as a last resort if a request arrives between a readiness check and semaphore exhaustion. -- **Horizontal Pod Autoscaler (HPA)** — The Wrapper Deployment can be configured with an HPA that scales replicas based on CPU/memory utilization or a custom metric such as the in-flight validation count exposed via a `/metrics` endpoint. Because the readiness probe already removes saturated pods from the Service, the HPA's scaling decisions and the readiness-driven traffic shifting work together: the HPA adds capacity while readiness prevents overload on existing pods. This allows the cluster to absorb demand spikes without requiring a centralized queue. -- **Resource limits and requests** — Each Wrapper pod should declare CPU and memory `requests` and `limits` that account for the peak resource usage of the CLI subprocess (the semaphore concurrency multiplied by the per-process footprint). This prevents a burst of validations from starving other workloads on the node. -- **NetworkPolicy** — A Kubernetes `NetworkPolicy` can restrict ingress to the Wrapper pods so that only Trustify pods (selected by label) are allowed to reach the `/api/v1/validate` endpoint. This provides a network-layer defense-in-depth on top of the OIDC-based authentication, ensuring that even if a valid token were leaked, it could not be used from outside the trusted namespace. -- **Service discovery** — Trustify references the Wrapper via its Kubernetes `Service` DNS name (e.g. `conforma-wrapper.trustify.svc.cluster.local`). This decouples Trustify from individual pod IPs and lets K8s load-balance requests across ready Wrapper replicas automatically. Combined with the readiness probe, Trustify never needs to be aware of individual pod capacity — the Service only routes to pods that have reported ready. -- **Secrets management** — The OIDC client secret and any policy-repo credentials should be mounted from Kubernetes `Secret` resources (or injected via an external secrets operator such as External Secrets or HashiCorp Vault Agent) rather than embedded in environment variable literals. This integrates with K8s RBAC to limit which pods and service accounts can access the sensitive material. +- **Interval**: Configurable via `CONFORMA_POLL_INTERVAL_SECONDS` (default: 15 seconds). +- **Batch processing**: On each tick, it fetches all `in_progress` validation rows and checks their status with the Conforma API. Requests are made concurrently (bounded by an internal semaphore to avoid overwhelming the Conforma API). +- **Timeout detection**: If a validation has been `in_progress` longer than its configured timeout, the poller transitions it to `failed` without querying the Conforma API. +- **Idempotency**: The poller uses conditional updates (`UPDATE ... WHERE status = 'in_progress' AND id = ?`) so it is safe to run from multiple Trustify replicas concurrently. +- **Error handling**: If the Conforma API is temporarily unreachable during a poll cycle, the poller logs the error and retries on the next tick. Transient API errors do not cause validation failures — only persistent unreachability beyond the timeout window does. #### Policy Management -When the policy.policy_type is "Conforma", the initial only policy type supported, the `policy` is using external references only and therefore Trustify does not cache policy content. - -Conforma fetches the policy at validation time from the git source specified in `policy.configuration.policy_ref`. +When the `policy.policy_type` is `"Conforma"`, Trustify stores only external references and does not cache policy content. Conforma fetches the policy at validation time from the git source specified in `policy.configuration.policy_ref`. The trade-off: validation always uses the latest policy content from the referenced branch or tag, but network failures or policy repo outages will cause execution errors. For private policy repositories, authentication credentials are stored in the `configuration` JSONB column and encrypted at rest using `ring::aead` (AES-256-GCM authenticated encryption); they are never logged. The `ring` crate is already a direct dependency of the project (used for digest hashing), so no new dependency is required. -The `policy_validation.policy_version` field records the policy commit hash or tag resolved from the `policy_ref` git source at validation time, enabling reproducibility and audit. -`policy_validation.summary.conforma_version`, which tracks the Conforma CLI tool version number (e.g., `v0.8.83`). +The `policy_validation.type_metadata.conforma_version` field records the Conforma version used for the validation, enabling reproducibility and audit. + +#### Kubernetes Deployment + +When both Trustify and Conforma are deployed on a Kubernetes cluster, native K8s primitives complement the application-level controls: + +- **Service discovery** — Trustify references the Conforma API via its Kubernetes `Service` DNS name (e.g. `conforma-api.trustify.svc.cluster.local`). This decouples Trustify from individual pod IPs and lets K8s load-balance requests across replicas. +- **Resource limits and requests** — Each Conforma API pod should declare CPU and memory `requests` and `limits` appropriate for its validation workload. +- **NetworkPolicy** — A Kubernetes `NetworkPolicy` can restrict ingress to the Conforma API pods so that only Trustify pods (selected by label) are allowed to reach the API. This provides network-layer defense-in-depth on top of token-based authentication. +- **Secrets management** — OIDC client secrets, API keys, and policy-repo credentials should be mounted from Kubernetes `Secret` resources (or injected via an external secrets operator) rather than embedded in environment variable literals. ## Future Work +#### Conforma API Availability + +This ADR's implementation is contingent on the Conforma REST API becoming available. The expected API contract documented above will be refined in collaboration with the Conforma team. Trustify's client adapter is designed behind a trait interface to absorb API contract changes with minimal impact on the rest of the codebase. + #### Validation on SBOM upload #### Multi-tenancy