diff --git a/ABI-FFI-README.md b/ABI-FFI-README.adoc similarity index 74% rename from ABI-FFI-README.md rename to ABI-FFI-README.adoc index f06f72cb..f1163e35 100644 --- a/ABI-FFI-README.md +++ b/ABI-FFI-README.adoc @@ -1,19 +1,22 @@ -{{~ Aditionally delete this line and fill out the template below ~}} +\{\{~ Aditionally delete this line and fill out the template below ~}} -# {{PROJECT}} ABI/FFI Documentation +== \{\{PROJECT}} ABI/FFI Documentation -## Overview +=== Overview -This library follows the **Hyperpolymath RSR Standard** for ABI and FFI design: +This library follows the *Hyperpolymath RSR Standard* for ABI and FFI +design: -- **ABI (Application Binary Interface)** defined in **Idris2** with formal proofs -- **FFI (Foreign Function Interface)** implemented in **Zig** for C compatibility -- **Generated C headers** bridge Idris2 ABI to Zig FFI -- **Any language** can call through standard C ABI +* *ABI (Application Binary Interface)* defined in *Idris2* with formal +proofs +* *FFI (Foreign Function Interface)* implemented in *Zig* for C +compatibility +* *Generated C headers* bridge Idris2 ABI to Zig FFI +* *Any language* can call through standard C ABI -## Architecture +=== Architecture -``` +.... ┌─────────────────────────────────────────────┐ │ ABI Definitions (Idris2) │ │ src/abi/ │ @@ -45,11 +48,11 @@ This library follows the **Hyperpolymath RSR Standard** for ABI and FFI design: │ Any Language via C ABI │ │ - Rust, ReScript, Julia, Python, etc. │ └─────────────────────────────────────────────┘ -``` +.... -## Directory Structure +=== Directory Structure -``` +.... {{project}}/ ├── src/ │ ├── abi/ # ABI definitions (Idris2) @@ -77,15 +80,17 @@ This library follows the **Hyperpolymath RSR Standard** for ABI and FFI design: ├── rust/ ├── rescript/ └── julia/ -``` +.... -## Why Idris2 for ABI? +=== Why Idris2 for ABI? -### 1. **Formal Verification** +==== 1. *Formal Verification* -Idris2's dependent types allow proving properties about the ABI at compile-time: +Idris2’s dependent types allow proving properties about the ABI at +compile-time: -```idris +[source,idris] +---- -- Prove struct size is correct public export exampleStructSize : HasSize ExampleStruct 16 @@ -97,13 +102,14 @@ fieldAligned : Divides 8 (offsetOf ExampleStruct.field) -- Prove ABI is platform-compatible public export abiCompatible : Compatible (ABI 1) (ABI 2) -``` +---- -### 2. **Type Safety** +==== 2. *Type Safety* Encode invariants that C/Zig cannot express: -```idris +[source,idris] +---- -- Non-null pointer guaranteed at type level data Handle : Type where MkHandle : (ptr : Bits64) -> {auto 0 nonNull : So (ptr /= 0)} -> Handle @@ -111,13 +117,14 @@ data Handle : Type where -- Array with length proof data Buffer : (n : Nat) -> Type where MkBuffer : Vect n Byte -> Buffer n -``` +---- -### 3. **Platform Abstraction** +==== 3. *Platform Abstraction* Platform-specific types with compile-time selection: -```idris +[source,idris] +---- CInt : Platform -> Type CInt Linux = Bits32 CInt Windows = Bits32 @@ -125,13 +132,14 @@ CInt Windows = Bits32 CSize : Platform -> Type CSize Linux = Bits64 CSize Windows = Bits64 -``` +---- -### 4. **Safe Evolution** +==== 4. *Safe Evolution* Prove that new ABI versions are backward-compatible: -```idris +[source,idris] +---- -- Compiler enforces compatibility abiUpgrade : ABI 1 -> ABI 2 abiUpgrade old = MkABI2 { @@ -140,71 +148,78 @@ abiUpgrade old = MkABI2 { -- Can add new fields new_features = defaults } -``` +---- -## Why Zig for FFI? +=== Why Zig for FFI? -### 1. **C ABI Compatibility** +==== 1. *C ABI Compatibility* Zig exports C-compatible functions naturally: -```zig +[source,zig] +---- export fn library_function(param: i32) i32 { return param * 2; } -``` +---- -### 2. **Memory Safety** +==== 2. *Memory Safety* Compile-time safety without runtime overhead: -```zig +[source,zig] +---- // Null check enforced at compile time const handle = init() orelse return error.InitFailed; defer free(handle); -``` +---- -### 3. **Cross-Compilation** +==== 3. *Cross-Compilation* Built-in cross-compilation to any platform: -```bash +[source,bash] +---- zig build -Dtarget=x86_64-linux zig build -Dtarget=aarch64-macos zig build -Dtarget=x86_64-windows -``` +---- -### 4. **Zero Dependencies** +==== 4. *Zero Dependencies* No runtime, no libc required (unless explicitly needed): -```zig +[source,zig] +---- // Minimal binary size pub const lib = @import("std"); // Only includes what you use -``` +---- -## Building +=== Building -### Build FFI Library +==== Build FFI Library -```bash +[source,bash] +---- cd ffi/zig zig build # Build debug zig build -Doptimize=ReleaseFast # Build optimized zig build test # Run tests -``` +---- -### Generate C Header from Idris2 ABI +==== Generate C Header from Idris2 ABI -```bash +[source,bash] +---- cd src/abi idris2 --cg c-header Types.idr -o ../../generated/abi/{{project}}.h -``` +---- -### Cross-Compile +==== Cross-Compile -```bash +[source,bash] +---- cd ffi/zig # Linux x86_64 @@ -215,13 +230,14 @@ zig build -Dtarget=aarch64-macos # Windows x86_64 zig build -Dtarget=x86_64-windows -``` +---- -## Usage +=== Usage -### From C +==== From C -```c +[source,c] +---- #include "{{project}}.h" int main() { @@ -237,16 +253,19 @@ int main() { {{project}}_free(handle); return 0; } -``` +---- Compile with: -```bash + +[source,bash] +---- gcc -o example example.c -l{{project}} -L./zig-out/lib -``` +---- -### From Idris2 +==== From Idris2 -```idris +[source,idris] +---- import {{PROJECT}}.ABI.Foreign main : IO () @@ -259,11 +278,12 @@ main = do free handle putStrLn "Success" -``` +---- -### From Rust +==== From Rust -```rust +[source,rust] +---- #[link(name = "{{project}}")] extern "C" { fn {{project}}_init() -> *mut std::ffi::c_void; @@ -282,11 +302,12 @@ fn main() { {{project}}_free(handle); } } -``` +---- -### From Julia +==== From Julia -```julia +[source,julia] +---- const lib{{project}} = "lib{{project}}" function init() @@ -312,27 +333,30 @@ try finally cleanup(handle) end -``` +---- -## Testing +=== Testing -### Unit Tests (Zig) +==== Unit Tests (Zig) -```bash +[source,bash] +---- cd ffi/zig zig build test -``` +---- -### Integration Tests +==== Integration Tests -```bash +[source,bash] +---- cd ffi/zig zig build test-integration -``` +---- -### ABI Verification (Idris2) +==== ABI Verification (Idris2) -```idris +[source,idris] +---- -- Compile-time verification %runElab verifyABI @@ -342,44 +366,44 @@ main = do verifyLayoutsCorrect verifyAlignmentsCorrect putStrLn "ABI verification passed" -``` +---- -## Contributing +=== Contributing When modifying the ABI/FFI: -1. **Update ABI first** (`src/abi/*.idr`) - - Modify type definitions - - Update proofs - - Ensure backward compatibility - -2. **Generate C header** - ```bash - idris2 --cg c-header src/abi/Types.idr -o generated/abi/{{project}}.h - ``` - -3. **Update FFI implementation** (`ffi/zig/src/main.zig`) - - Implement new functions - - Match ABI types exactly - -4. **Add tests** - - Unit tests in Zig - - Integration tests - - ABI verification tests - -5. **Update documentation** - - Function signatures - - Usage examples - - Migration guide (if breaking changes) - -## License +[arabic] +. *Update ABI first* (`+src/abi/*.idr+`) +* Modify type definitions +* Update proofs +* Ensure backward compatibility +. *Generate C header* ++ +[source,bash] +---- +idris2 --cg c-header src/abi/Types.idr -o generated/abi/{{project}}.h +---- +. *Update FFI implementation* (`+ffi/zig/src/main.zig+`) +* Implement new functions +* Match ABI types exactly +. *Add tests* +* Unit tests in Zig +* Integration tests +* ABI verification tests +. *Update documentation* +* Function signatures +* Usage examples +* Migration guide (if breaking changes) + +=== License MPL-2.0 -## See Also +=== See Also -- [Idris2 Documentation](https://idris2.readthedocs.io) -- [Zig Documentation](https://ziglang.org/documentation/master/) -- [Rhodium Standard Repositories](https://github.com/hyperpolymath/rhodium-standard-repositories) -- [FFI Migration Guide](../ffi-migration-guide.md) -- [ABI Migration Guide](../abi-migration-guide.md) +* https://idris2.readthedocs.io[Idris2 Documentation] +* https://ziglang.org/documentation/master/[Zig Documentation] +* https://github.com/hyperpolymath/rhodium-standard-repositories[Rhodium +Standard Repositories] +* link:../ffi-migration-guide.md[FFI Migration Guide] +* link:../abi-migration-guide.md[ABI Migration Guide] diff --git a/ARCHITECTURE.adoc b/ARCHITECTURE.adoc new file mode 100644 index 00000000..1c0a7a69 --- /dev/null +++ b/ARCHITECTURE.adoc @@ -0,0 +1,48 @@ +== Architecture + +=== Overview + +This repository follows a modular, maintainable architecture designed +for clarity, scalability, and long-term sustainability. + +=== Directory Structure + +.... +. +├── src/ # Source code +├── tests/ # Test suites +├── docs/ # Documentation +├── scripts/ # Utility scripts +├── config/ # Configuration files +├── LICENSE # License file +├── LICENSES/ # Full license texts +└── README.adoc # Project documentation +.... + +=== Design Principles + +* *Separation of Concerns*: Each module has a single responsibility +* *Testability*: Code is written to be easily testable +* *Documentation*: All public APIs are documented +* *Configuration*: Environment-specific settings are externalized + +=== Dependencies + +* External dependencies are minimized and clearly declared +* Version pinning is used for reproducibility + +=== Security Considerations + +* Sensitive data is never committed to the repository +* Secrets are managed through environment variables or secure vaults +* Regular dependency audits are performed + +=== Maintainability + +* Code follows consistent style guidelines +* Pull requests require review and CI checks +* Issues and discussions are tracked transparently + +''''' + +_Last updated: 2026-07-18_ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md deleted file mode 100644 index 607e3d8c..00000000 --- a/ARCHITECTURE.md +++ /dev/null @@ -1,47 +0,0 @@ -# Architecture - -## Overview - -This repository follows a modular, maintainable architecture designed for clarity, scalability, and long-term sustainability. - -## Directory Structure - -``` -. -├── src/ # Source code -├── tests/ # Test suites -├── docs/ # Documentation -├── scripts/ # Utility scripts -├── config/ # Configuration files -├── LICENSE # License file -├── LICENSES/ # Full license texts -└── README.adoc # Project documentation -``` - -## Design Principles - -- **Separation of Concerns**: Each module has a single responsibility -- **Testability**: Code is written to be easily testable -- **Documentation**: All public APIs are documented -- **Configuration**: Environment-specific settings are externalized - -## Dependencies - -- External dependencies are minimized and clearly declared -- Version pinning is used for reproducibility - -## Security Considerations - -- Sensitive data is never committed to the repository -- Secrets are managed through environment variables or secure vaults -- Regular dependency audits are performed - -## Maintainability - -- Code follows consistent style guidelines -- Pull requests require review and CI checks -- Issues and discussions are tracked transparently - ---- - -*Last updated: 2026-07-18* diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc new file mode 100644 index 00000000..90ca682b --- /dev/null +++ b/CHANGELOG.adoc @@ -0,0 +1,57 @@ +== Changelog + +All notable changes to `+verisimdb-data+` will be documented in this +file. + +This file is generated from conventional commits by the +https://github.com/hyperpolymath/standards/blob/main/.github/workflows/changelog-reusable.yml[`+changelog-reusable.yml+`] +workflow (`+hyperpolymath/standards#206+`). Adopt the workflow in this +repo’s CI to keep this file in sync automatically — see +https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml[`+templates/cliff.toml+`] +for the canonical config. + +The format follows https://keepachangelog.com/en/1.1.0/[Keep a +Changelog]; this project aims to follow +https://semver.org/spec/v2.0.0.html[Semantic Versioning]. + +=== [Unreleased] + +==== Added + +* feat(crg): add crg-grade and crg-badge justfile recipes +* feat: add self-health VeriSimDB Zig module to close observability loop +* feat: migrate generate-summaries from Python to Julia +* feat: add stapeln.toml container definition +* feat: add UX Justfile with doctor, tour, help-me, assail recipes +* feat: deploy UX Manifesto infrastructure +* feat: initialize verisimdb-data as standalone repo + +==== Fixed + +* fix(ci): Phase-2 fleet submission must not fail the security gate (#9) +* fix(ci): hypatia-scan workdir ($\{\{ env.HOME }} resolves empty) (#8) + +==== Documentation + +* docs: add M2 estate audit report (2026-04-04) +* docs: add TOPOLOGY.md +* docs: add TEST-NEEDS.md (CRG C) +* docs: add EXPLAINME.adoc — prove-it file backing README claims + +==== CI + +* ci: bump actions/upload-artifact SHA to current v4 (#3) +* ci: SHA-pin hyperpolymath validate-actions in dogfood-gate +* ci: fix workflow-linter YAML parse error + self-flag bug +* ci: restore Dependabot security path + wire auto-merge +* ci: deploy dogfood-gate, fix hypatia-scan, add pre-commit hooks + +=== Pre-history + +Prior commits to this file’s introduction are recorded in git history +but not formally classified into Keep-a-Changelog sections. To backfill, +run `+git cliff -o CHANGELOG.md+` locally using the canonical +https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml[`+cliff.toml+`] +— this is one-shot mechanical work. + +''''' diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index 892c1958..00000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,57 +0,0 @@ - - -# Changelog - -All notable changes to `verisimdb-data` will be documented in this file. - -This file is generated from conventional commits by the -[`changelog-reusable.yml`](https://github.com/hyperpolymath/standards/blob/main/.github/workflows/changelog-reusable.yml) -workflow (`hyperpolymath/standards#206`). Adopt the workflow in this repo's CI to keep this file in sync automatically — see -[`templates/cliff.toml`](https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml) -for the canonical config. - -The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); -this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - -## [Unreleased] - -### Added - -- feat(crg): add crg-grade and crg-badge justfile recipes -- feat: add self-health VeriSimDB Zig module to close observability loop -- feat: migrate generate-summaries from Python to Julia -- feat: add stapeln.toml container definition -- feat: add UX Justfile with doctor, tour, help-me, assail recipes -- feat: deploy UX Manifesto infrastructure -- feat: initialize verisimdb-data as standalone repo - -### Fixed - -- fix(ci): Phase-2 fleet submission must not fail the security gate (#9) -- fix(ci): hypatia-scan workdir (${{ env.HOME }} resolves empty) (#8) - -### Documentation - -- docs: add M2 estate audit report (2026-04-04) -- docs: add TOPOLOGY.md -- docs: add TEST-NEEDS.md (CRG C) -- docs: add EXPLAINME.adoc — prove-it file backing README claims - -### CI - -- ci: bump actions/upload-artifact SHA to current v4 (#3) -- ci: SHA-pin hyperpolymath validate-actions in dogfood-gate -- ci: fix workflow-linter YAML parse error + self-flag bug -- ci: restore Dependabot security path + wire auto-merge -- ci: deploy dogfood-gate, fix hypatia-scan, add pre-commit hooks - -## Pre-history - -Prior commits to this file's introduction are recorded in git history but not formally classified into Keep-a-Changelog sections. To backfill, run `git cliff -o CHANGELOG.md` locally using the canonical [`cliff.toml`](https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml) — this is one-shot mechanical work. - ---- - - diff --git a/CODE_OF_CONDUCT.adoc b/CODE_OF_CONDUCT.adoc new file mode 100644 index 00000000..4fedda1c --- /dev/null +++ b/CODE_OF_CONDUCT.adoc @@ -0,0 +1,339 @@ +== Code of Conduct + +=== Our Pledge + +We as members, contributors, and leaders pledge to make participation in +Nextgen Databases a harassment-free experience for everyone, regardless +of age, body size, visible or invisible disability, ethnicity, sex +characteristics, gender identity and expression, level of experience, +education, socio-economic status, nationality, personal appearance, +race, caste, colour, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, +welcoming, diverse, inclusive, and healthy community. + +We recognise that a thriving open source community requires +*psychological safety* — an environment where people can contribute, ask +questions, make mistakes, and learn without fear of ridicule or +retaliation. + +''''' + +=== Our Standards + +==== Expected Behaviour + +The following behaviours contribute to a positive environment: + +*Communication* - Using welcoming and inclusive language - Being +respectful of differing viewpoints and experiences - Giving and +gracefully accepting constructive feedback - Assuming good intent while +addressing impact - Communicating clearly and patiently, especially with +newcomers + +*Collaboration* - Focusing on what is best for the community - Showing +empathy and kindness toward other community members - Being +collaborative rather than competitive - Mentoring and supporting less +experienced contributors - Celebrating others’ contributions and +successes + +*Professionalism* - Accepting responsibility and apologising to those +affected by our mistakes - Learning from the experience and avoiding +repetition - Respecting others’ time and attention - Staying on topic in +project spaces - Following project guidelines and conventions + +*Accessibility* - Using plain language and avoiding unnecessary jargon - +Providing alt text for images and transcripts for audio/video - Being +patient with those using assistive technologies - Accommodating +different communication styles and needs - Recognising that not everyone +communicates the same way + +==== Unacceptable Behaviour + +The following behaviours are considered harassment and are unacceptable: + +*Harassment* - The use of sexualised language or imagery, and sexual +attention or advances of any kind - Trolling, insulting or derogatory +comments, and personal or political attacks - Public or private +harassment - Deliberate intimidation, stalking, or following (online or +in-person) - Unwelcome physical contact or simulated physical contact +(e.g., emoji) - Sustained disruption of talks, events, or online +discussions + +*Discrimination* - Discriminatory jokes and language - Posting or +threatening to post others’ personally identifying information +("`doxing`") - Advocating for, or encouraging, any of the above +behaviour - Microaggressions — subtle, often unintentional, +discriminatory comments or actions + +*Professional Misconduct* - Publishing others’ private information +without explicit permission - Misrepresenting affiliation or +contributions - Plagiarism or claiming credit for others’ work - +Retaliating against anyone who reports a Code of Conduct violation - +Other conduct which could reasonably be considered inappropriate in a +professional setting + +==== Grey Areas + +Some situations require judgement. When uncertain: + +* *Intent vs Impact*: Good intentions do not excuse harmful impact. +Focus on making things right. +* *Power Dynamics*: Those with more power (maintainers, employers, +experienced contributors) must be especially mindful of their impact. +* *Cultural Differences*: What’s acceptable varies by culture. When in +doubt, err on the side of caution and ask. +* *Humour*: Jokes at others’ expense are rarely funny to everyone. Punch +up, not down. + +''''' + +=== Scope + +This Code of Conduct applies within all community spaces, including: + +*Online Spaces* - Repository discussions, issues, and pull/merge +requests - Project chat channels (Matrix, Discord, Slack, IRC) - Mailing +lists and forums - Social media when representing the project - Video +calls and virtual meetings + +*In-Person Spaces* - Conferences, meetups, and events - Workshops and +training sessions - Any gathering where you represent the project + +*Representation* This Code of Conduct also applies when an individual is +officially representing the community in public spaces. Examples +include: + +* Using an official project email address +* Posting via an official social media account +* Acting as an appointed representative at an event +* Speaking on behalf of the project + +''''' + +=== Enforcement + +==== Reporting + +If you experience or witness unacceptable behaviour, or have any other +concerns, please report it as soon as possible. + +*How to Report* + +[width="99%",cols="30%,33%,37%",options="header",] +|=== +|Method |Details |Best For +|*Email* |\{\{CONDUCT_EMAIL}} |Detailed reports, sensitive matters + +|*Private Message* |Contact any maintainer directly |Quick questions, +minor issues + +|*Anonymous Form* |[Link to form if available] |When you need anonymity +|=== + +*What to Include* + +* Your contact information (unless anonymous) +* Names/usernames of those involved +* Description of what happened +* When and where it occurred +* Any witnesses +* Any supporting evidence (screenshots, links) +* How you would like us to respond (if you have a preference) + +*What Happens Next* + +[arabic] +. You will receive acknowledgment within *\{\{RESPONSE_TIME}}* +. The \{\{CONDUCT_TEAM}} will review the report +. We may ask for additional information +. We will determine appropriate action +. We will inform you of the outcome (respecting others’ privacy) + +==== Confidentiality + +All reports will be handled with discretion: + +* Reporter identity is protected by default +* Details are shared only with those who need to know +* We will ask before naming you in any communication +* Anonymous reports are accepted and investigated + +==== Conflicts of Interest + +If a \{\{CONDUCT_TEAM}} member is involved in an incident: + +* They will recuse themselves from the process +* Another maintainer or external party will handle the report +* We will disclose any potential conflicts + +''''' + +=== Enforcement Guidelines + +The \{\{CONDUCT_TEAM}} will follow these guidelines in determining +consequences: + +==== 1. Correction + +*Community Impact*: Use of inappropriate language or other behaviour +deemed unprofessional or unwelcome. + +*Consequence*: A private, written warning providing clarity around the +nature of the violation and an explanation of why the behaviour was +inappropriate. A public apology may be requested. + +*Duration*: Immediate + +==== 2. Warning + +*Community Impact*: A violation through a single incident or series of +actions. + +*Consequence*: A warning with consequences for continued behaviour. No +interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, for a specified period. This +includes avoiding interactions in community spaces as well as external +channels like social media. Violating these terms may lead to a +temporary or permanent ban. + +*Duration*: 1-4 weeks + +==== 3. Temporary Ban + +*Community Impact*: A serious violation of community standards, +including sustained inappropriate behaviour. + +*Consequence*: A temporary ban from any sort of interaction or public +communication with the community for a specified period. No public or +private interaction with the people involved, including unsolicited +interaction with those enforcing the Code of Conduct, is allowed during +this period. Violating these terms may lead to a permanent ban. + +*Duration*: 1-6 months + +==== 4. Permanent Ban + +*Community Impact*: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behaviour, harassment of an +individual, or aggression toward or disparagement of classes of +individuals. + +*Consequence*: A permanent ban from any sort of public interaction +within the community. + +*Duration*: Permanent (with appeal rights after 12 months) + +==== Enforcement Across Perimeters + +For contributors with elevated access (Perimeter 2 or 1): + +[cols=",",options="header",] +|=== +|Level |Additional Consequence +|Correction |Noted in contributor record +|Warning |Access privileges may be temporarily reduced +|Temporary Ban |Access reduced to Perimeter 3 for ban duration +|Permanent Ban |All access revoked +|=== + +''''' + +=== Appeals + +If you believe an enforcement decision was made in error: + +[arabic] +. *Wait 7 days* after the decision (cooling-off period) +. *Email* \{\{CONDUCT_EMAIL}} with subject line "`Appeal: [Original +Report ID]`" +. *Explain* why you believe the decision should be reconsidered +. *Provide* any new information not previously available + +*Appeals Process* + +* Appeals are reviewed by a different \{\{CONDUCT_TEAM}} member than the +original +* You will receive a response within 14 days +* The appeals decision is final +* You may only appeal once per incident + +*Grounds for Appeal* + +* Procedural errors in the original investigation +* New evidence not previously available +* Disproportionate response to the violation +* Misunderstanding of facts + +''''' + +=== Supporting Those Who Report + +We are committed to supporting those who report violations: + +*We Will* - Believe and take all reports seriously - Respect your +privacy and confidentiality preferences - Keep you informed of progress +(if you wish) - Take steps to protect you from retaliation - Provide +resources if you need support + +*We Will Not* - Require you to confront the person directly - Dismiss +reports without investigation - Reveal your identity without consent - +Tolerate retaliation against reporters - Rush you to make decisions + +''''' + +=== Prevention + +Beyond enforcement, we actively work to prevent issues: + +*Onboarding* - All contributors are expected to read this Code of +Conduct - Perimeter 2 applicants must confirm they’ve read and +understood it - Maintainers receive additional training on enforcement + +*Culture* - We model the behaviour we expect - We intervene early when +we see potential issues - We thank people for positive contributions - +We create opportunities for diverse voices + +*Review* - This Code of Conduct is reviewed annually - Community +feedback is welcomed - Changes are communicated clearly + +''''' + +=== Acknowledgments + +This Code of Conduct is adapted from: + +* https://www.contributor-covenant.org/[Contributor Covenant], version +2.1 +* https://www.djangoproject.com/conduct/[Django Code of Conduct] +* https://www.rust-lang.org/policies/code-of-conduct[Rust Code of +Conduct] +* https://www.python.org/psf/conduct/[Python Community Code of Conduct] + +We thank these communities for their leadership in creating welcoming +spaces. + +''''' + +=== Questions? + +If you have questions about this Code of Conduct: + +* Open a +https://github.com/hyperpolymath/nextgen-databases/discussions[Discussion] +(for general questions) +* Email \{\{CONDUCT_EMAIL}} (for private questions) +* Contact any maintainer directly + +''''' + +=== Summary + +*Be kind. Be respectful. Be collaborative.* + +We’re all here because we care about this project. Let’s make it a place +where everyone can do their best work. + +''''' + +Last updated: 2026 · Based on Contributor Covenant 2.1 diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md deleted file mode 100644 index ab314f8c..00000000 --- a/CODE_OF_CONDUCT.md +++ /dev/null @@ -1,327 +0,0 @@ -# Code of Conduct - - - -## Our Pledge - -We as members, contributors, and leaders pledge to make participation in Nextgen Databases a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, colour, religion, or sexual identity and orientation. - -We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. - -We recognise that a thriving open source community requires **psychological safety** — an environment where people can contribute, ask questions, make mistakes, and learn without fear of ridicule or retaliation. - ---- - -## Our Standards - -### Expected Behaviour - -The following behaviours contribute to a positive environment: - -**Communication** -- Using welcoming and inclusive language -- Being respectful of differing viewpoints and experiences -- Giving and gracefully accepting constructive feedback -- Assuming good intent while addressing impact -- Communicating clearly and patiently, especially with newcomers - -**Collaboration** -- Focusing on what is best for the community -- Showing empathy and kindness toward other community members -- Being collaborative rather than competitive -- Mentoring and supporting less experienced contributors -- Celebrating others' contributions and successes - -**Professionalism** -- Accepting responsibility and apologising to those affected by our mistakes -- Learning from the experience and avoiding repetition -- Respecting others' time and attention -- Staying on topic in project spaces -- Following project guidelines and conventions - -**Accessibility** -- Using plain language and avoiding unnecessary jargon -- Providing alt text for images and transcripts for audio/video -- Being patient with those using assistive technologies -- Accommodating different communication styles and needs -- Recognising that not everyone communicates the same way - -### Unacceptable Behaviour - -The following behaviours are considered harassment and are unacceptable: - -**Harassment** -- The use of sexualised language or imagery, and sexual attention or advances of any kind -- Trolling, insulting or derogatory comments, and personal or political attacks -- Public or private harassment -- Deliberate intimidation, stalking, or following (online or in-person) -- Unwelcome physical contact or simulated physical contact (e.g., emoji) -- Sustained disruption of talks, events, or online discussions - -**Discrimination** -- Discriminatory jokes and language -- Posting or threatening to post others' personally identifying information ("doxing") -- Advocating for, or encouraging, any of the above behaviour -- Microaggressions — subtle, often unintentional, discriminatory comments or actions - -**Professional Misconduct** -- Publishing others' private information without explicit permission -- Misrepresenting affiliation or contributions -- Plagiarism or claiming credit for others' work -- Retaliating against anyone who reports a Code of Conduct violation -- Other conduct which could reasonably be considered inappropriate in a professional setting - -### Grey Areas - -Some situations require judgement. When uncertain: - -- **Intent vs Impact**: Good intentions do not excuse harmful impact. Focus on making things right. -- **Power Dynamics**: Those with more power (maintainers, employers, experienced contributors) must be especially mindful of their impact. -- **Cultural Differences**: What's acceptable varies by culture. When in doubt, err on the side of caution and ask. -- **Humour**: Jokes at others' expense are rarely funny to everyone. Punch up, not down. - ---- - -## Scope - -This Code of Conduct applies within all community spaces, including: - -**Online Spaces** -- Repository discussions, issues, and pull/merge requests -- Project chat channels (Matrix, Discord, Slack, IRC) -- Mailing lists and forums -- Social media when representing the project -- Video calls and virtual meetings - -**In-Person Spaces** -- Conferences, meetups, and events -- Workshops and training sessions -- Any gathering where you represent the project - -**Representation** -This Code of Conduct also applies when an individual is officially representing the community in public spaces. Examples include: - -- Using an official project email address -- Posting via an official social media account -- Acting as an appointed representative at an event -- Speaking on behalf of the project - ---- - -## Enforcement - -### Reporting - -If you experience or witness unacceptable behaviour, or have any other concerns, please report it as soon as possible. - -**How to Report** - -| Method | Details | Best For | -|--------|---------|----------| -| **Email** | {{CONDUCT_EMAIL}} | Detailed reports, sensitive matters | -| **Private Message** | Contact any maintainer directly | Quick questions, minor issues | -| **Anonymous Form** | [Link to form if available] | When you need anonymity | - -**What to Include** - -- Your contact information (unless anonymous) -- Names/usernames of those involved -- Description of what happened -- When and where it occurred -- Any witnesses -- Any supporting evidence (screenshots, links) -- How you would like us to respond (if you have a preference) - -**What Happens Next** - -1. You will receive acknowledgment within **{{RESPONSE_TIME}}** -2. The {{CONDUCT_TEAM}} will review the report -3. We may ask for additional information -4. We will determine appropriate action -5. We will inform you of the outcome (respecting others' privacy) - -### Confidentiality - -All reports will be handled with discretion: - -- Reporter identity is protected by default -- Details are shared only with those who need to know -- We will ask before naming you in any communication -- Anonymous reports are accepted and investigated - -### Conflicts of Interest - -If a {{CONDUCT_TEAM}} member is involved in an incident: - -- They will recuse themselves from the process -- Another maintainer or external party will handle the report -- We will disclose any potential conflicts - ---- - -## Enforcement Guidelines - -The {{CONDUCT_TEAM}} will follow these guidelines in determining consequences: - -### 1. Correction - -**Community Impact**: Use of inappropriate language or other behaviour deemed unprofessional or unwelcome. - -**Consequence**: A private, written warning providing clarity around the nature of the violation and an explanation of why the behaviour was inappropriate. A public apology may be requested. - -**Duration**: Immediate - -### 2. Warning - -**Community Impact**: A violation through a single incident or series of actions. - -**Consequence**: A warning with consequences for continued behaviour. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban. - -**Duration**: 1-4 weeks - -### 3. Temporary Ban - -**Community Impact**: A serious violation of community standards, including sustained inappropriate behaviour. - -**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban. - -**Duration**: 1-6 months - -### 4. Permanent Ban - -**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behaviour, harassment of an individual, or aggression toward or disparagement of classes of individuals. - -**Consequence**: A permanent ban from any sort of public interaction within the community. - -**Duration**: Permanent (with appeal rights after 12 months) - -### Enforcement Across Perimeters - -For contributors with elevated access (Perimeter 2 or 1): - -| Level | Additional Consequence | -|-------|----------------------| -| Correction | Noted in contributor record | -| Warning | Access privileges may be temporarily reduced | -| Temporary Ban | Access reduced to Perimeter 3 for ban duration | -| Permanent Ban | All access revoked | - ---- - -## Appeals - -If you believe an enforcement decision was made in error: - -1. **Wait 7 days** after the decision (cooling-off period) -2. **Email** {{CONDUCT_EMAIL}} with subject line "Appeal: [Original Report ID]" -3. **Explain** why you believe the decision should be reconsidered -4. **Provide** any new information not previously available - -**Appeals Process** - -- Appeals are reviewed by a different {{CONDUCT_TEAM}} member than the original -- You will receive a response within 14 days -- The appeals decision is final -- You may only appeal once per incident - -**Grounds for Appeal** - -- Procedural errors in the original investigation -- New evidence not previously available -- Disproportionate response to the violation -- Misunderstanding of facts - ---- - -## Supporting Those Who Report - -We are committed to supporting those who report violations: - -**We Will** -- Believe and take all reports seriously -- Respect your privacy and confidentiality preferences -- Keep you informed of progress (if you wish) -- Take steps to protect you from retaliation -- Provide resources if you need support - -**We Will Not** -- Require you to confront the person directly -- Dismiss reports without investigation -- Reveal your identity without consent -- Tolerate retaliation against reporters -- Rush you to make decisions - ---- - -## Prevention - -Beyond enforcement, we actively work to prevent issues: - -**Onboarding** -- All contributors are expected to read this Code of Conduct -- Perimeter 2 applicants must confirm they've read and understood it -- Maintainers receive additional training on enforcement - -**Culture** -- We model the behaviour we expect -- We intervene early when we see potential issues -- We thank people for positive contributions -- We create opportunities for diverse voices - -**Review** -- This Code of Conduct is reviewed annually -- Community feedback is welcomed -- Changes are communicated clearly - ---- - -## Acknowledgments - -This Code of Conduct is adapted from: - -- [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1 -- [Django Code of Conduct](https://www.djangoproject.com/conduct/) -- [Rust Code of Conduct](https://www.rust-lang.org/policies/code-of-conduct) -- [Python Community Code of Conduct](https://www.python.org/psf/conduct/) - -We thank these communities for their leadership in creating welcoming spaces. - ---- - -## Questions? - -If you have questions about this Code of Conduct: - -- Open a [Discussion](https://github.com/hyperpolymath/nextgen-databases/discussions) (for general questions) -- Email {{CONDUCT_EMAIL}} (for private questions) -- Contact any maintainer directly - ---- - -## Summary - -**Be kind. Be respectful. Be collaborative.** - -We're all here because we care about this project. Let's make it a place where everyone can do their best work. - ---- - -Last updated: 2026 · Based on Contributor Covenant 2.1 diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc new file mode 100644 index 00000000..e9178b36 --- /dev/null +++ b/CONTRIBUTING.adoc @@ -0,0 +1,109 @@ +== Clone the repository + +git clone https://github.com/hyperpolymath/nextgen-databases.git cd +nextgen-databases + +== Using Nix (recommended for reproducibility) + +nix develop + +== Or using toolbox/distrobox + +toolbox create nextgen-databases-dev toolbox enter nextgen-databases-dev +# Install dependencies manually + +== Verify setup + +just check # or: cargo check / mix compile / etc. just test # Run test +suite + +.... + +### Repository Structure +.... + +nextgen-databases/ ├── src/ # Source code (Perimeter 1-2) ├── lib/ # +Library code (Perimeter 1-2) ├── extensions/ # Extensions (Perimeter 2) +├── plugins/ # Plugins (Perimeter 2) ├── tools/ # Tooling (Perimeter 2) +├── docs/ # Documentation (Perimeter 3) │ ├── architecture/ # ADRs, +specs (Perimeter 2) │ └── proposals/ # RFCs (Perimeter 3) ├── examples/ +# Examples (Perimeter 3) ├── spec/ # Spec tests (Perimeter 3) ├── tests/ +# Test suite (Perimeter 2-3) ├── .well-known/ # Protocol files +(Perimeter 1-3) ├── .github/ # GitHub config (Perimeter 1) │ ├── +ISSUE_TEMPLATE/ │ └── workflows/ ├── CHANGELOG.md ├── CODE_OF_CONDUCT.md +├── CONTRIBUTING.md # This file ├── GOVERNANCE.md ├── LICENSE ├── +MAINTAINERS.md ├── README.adoc ├── SECURITY.md ├── flake.nix # Nix flake +(Perimeter 1) └── Justfile # Task runner (Perimeter 1) + +.... + +--- + +## How to Contribute + +### Reporting Bugs + +**Before reporting**: +1. Search existing issues +2. Check if it's already fixed in `main` +3. Determine which perimeter the bug affects + +**When reporting**: + +Use the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md) and include: + +- Clear, descriptive title +- Environment details (OS, versions, toolchain) +- Steps to reproduce +- Expected vs actual behaviour +- Logs, screenshots, or minimal reproduction + +### Suggesting Features + +**Before suggesting**: +1. Check the [roadmap](ROADMAP.md) if available +2. Search existing issues and discussions +3. Consider which perimeter the feature belongs to + +**When suggesting**: + +Use the [feature request template](.github/ISSUE_TEMPLATE/feature_request.md) and include: + +- Problem statement (what pain point does this solve?) +- Proposed solution +- Alternatives considered +- Which perimeter this affects + +### Your First Contribution + +Look for issues labelled: + +- [`good first issue`](https://github.com/hyperpolymath/nextgen-databases/labels/good%20first%20issue) — Simple Perimeter 3 tasks +- [`help wanted`](https://github.com/hyperpolymath/nextgen-databases/labels/help%20wanted) — Community help needed +- [`documentation`](https://github.com/hyperpolymath/nextgen-databases/labels/documentation) — Docs improvements +- [`perimeter-3`](https://github.com/hyperpolymath/nextgen-databases/labels/perimeter-3) — Community sandbox scope + +--- + +## Development Workflow + +### Branch Naming +.... + +docs/short-description # Documentation (P3) test/what-added # Test +additions (P3) feat/short-description # New features (P2) +fix/issue-number-description # Bug fixes (P2) refactor/what-changed # +Code improvements (P2) security/what-fixed # Security fixes (P1-2) + +.... + +### Commit Messages + +We follow [Conventional Commits](https://www.conventionalcommits.org/): +.... + +(): + +{empty}[optional body] + +{empty}[optional footer] diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index bd6f5a5e..00000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,116 +0,0 @@ -# Clone the repository -git clone https://github.com/hyperpolymath/nextgen-databases.git -cd nextgen-databases - -# Using Nix (recommended for reproducibility) -nix develop - -# Or using toolbox/distrobox -toolbox create nextgen-databases-dev -toolbox enter nextgen-databases-dev -# Install dependencies manually - -# Verify setup -just check # or: cargo check / mix compile / etc. -just test # Run test suite -``` - -### Repository Structure -``` -nextgen-databases/ -├── src/ # Source code (Perimeter 1-2) -├── lib/ # Library code (Perimeter 1-2) -├── extensions/ # Extensions (Perimeter 2) -├── plugins/ # Plugins (Perimeter 2) -├── tools/ # Tooling (Perimeter 2) -├── docs/ # Documentation (Perimeter 3) -│ ├── architecture/ # ADRs, specs (Perimeter 2) -│ └── proposals/ # RFCs (Perimeter 3) -├── examples/ # Examples (Perimeter 3) -├── spec/ # Spec tests (Perimeter 3) -├── tests/ # Test suite (Perimeter 2-3) -├── .well-known/ # Protocol files (Perimeter 1-3) -├── .github/ # GitHub config (Perimeter 1) -│ ├── ISSUE_TEMPLATE/ -│ └── workflows/ -├── CHANGELOG.md -├── CODE_OF_CONDUCT.md -├── CONTRIBUTING.md # This file -├── GOVERNANCE.md -├── LICENSE -├── MAINTAINERS.md -├── README.adoc -├── SECURITY.md -├── flake.nix # Nix flake (Perimeter 1) -└── Justfile # Task runner (Perimeter 1) -``` - ---- - -## How to Contribute - -### Reporting Bugs - -**Before reporting**: -1. Search existing issues -2. Check if it's already fixed in `main` -3. Determine which perimeter the bug affects - -**When reporting**: - -Use the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md) and include: - -- Clear, descriptive title -- Environment details (OS, versions, toolchain) -- Steps to reproduce -- Expected vs actual behaviour -- Logs, screenshots, or minimal reproduction - -### Suggesting Features - -**Before suggesting**: -1. Check the [roadmap](ROADMAP.md) if available -2. Search existing issues and discussions -3. Consider which perimeter the feature belongs to - -**When suggesting**: - -Use the [feature request template](.github/ISSUE_TEMPLATE/feature_request.md) and include: - -- Problem statement (what pain point does this solve?) -- Proposed solution -- Alternatives considered -- Which perimeter this affects - -### Your First Contribution - -Look for issues labelled: - -- [`good first issue`](https://github.com/hyperpolymath/nextgen-databases/labels/good%20first%20issue) — Simple Perimeter 3 tasks -- [`help wanted`](https://github.com/hyperpolymath/nextgen-databases/labels/help%20wanted) — Community help needed -- [`documentation`](https://github.com/hyperpolymath/nextgen-databases/labels/documentation) — Docs improvements -- [`perimeter-3`](https://github.com/hyperpolymath/nextgen-databases/labels/perimeter-3) — Community sandbox scope - ---- - -## Development Workflow - -### Branch Naming -``` -docs/short-description # Documentation (P3) -test/what-added # Test additions (P3) -feat/short-description # New features (P2) -fix/issue-number-description # Bug fixes (P2) -refactor/what-changed # Code improvements (P2) -security/what-fixed # Security fixes (P1-2) -``` - -### Commit Messages - -We follow [Conventional Commits](https://www.conventionalcommits.org/): -``` -(): - -[optional body] - -[optional footer] diff --git a/GOVERNANCE.adoc b/GOVERNANCE.adoc new file mode 100644 index 00000000..9b836fb2 --- /dev/null +++ b/GOVERNANCE.adoc @@ -0,0 +1,60 @@ +== Governance + +=== Overview + +This project is governed by the following principles and structures to +ensure transparent, inclusive, and effective decision-making. + +=== Roles and Responsibilities + +==== Maintainers + +Maintainers are responsible for: - Reviewing and merging pull requests - +Managing releases and versioning - Ensuring code quality and standards - +Triaging issues and bug reports - Community engagement and support + +==== Contributors + +Contributors are expected to: - Follow the code of conduct - Submit +well-documented pull requests - Write tests for new functionality - +Maintain existing tests - Update documentation as needed + +=== Decision Making + +==== Minor Changes + +* Can be made by any maintainer +* Include bug fixes, documentation updates, dependency updates + +==== Major Changes + +* Require discussion in issues or pull requests +* Include new features, architectural changes, API changes +* Need approval from at least 2 maintainers + +==== Breaking Changes + +* Require RFC (Request for Comments) process +* Need approval from majority of maintainers +* Must include migration guide + +=== Code of Conduct + +All participants are expected to follow our Code of Conduct. Violations +can be reported to the maintainers. + +=== Communication + +* *Issues*: For bug reports and feature requests +* *Discussions*: For questions and general discussion +* *Pull Requests*: For code contributions + +=== Licensing + +All contributions are made under the terms of the repository’s LICENSE +file. By submitting a pull request, you agree to license your +contributions accordingly. + +''''' + +_Last updated: 2026-07-18_ diff --git a/GOVERNANCE.md b/GOVERNANCE.md deleted file mode 100644 index e27364c7..00000000 --- a/GOVERNANCE.md +++ /dev/null @@ -1,60 +0,0 @@ -# Governance - -## Overview - -This project is governed by the following principles and structures to ensure transparent, inclusive, and effective decision-making. - -## Roles and Responsibilities - -### Maintainers - -Maintainers are responsible for: -- Reviewing and merging pull requests -- Managing releases and versioning -- Ensuring code quality and standards -- Triaging issues and bug reports -- Community engagement and support - -### Contributors - -Contributors are expected to: -- Follow the code of conduct -- Submit well-documented pull requests -- Write tests for new functionality -- Maintain existing tests -- Update documentation as needed - -## Decision Making - -### Minor Changes -- Can be made by any maintainer -- Include bug fixes, documentation updates, dependency updates - -### Major Changes -- Require discussion in issues or pull requests -- Include new features, architectural changes, API changes -- Need approval from at least 2 maintainers - -### Breaking Changes -- Require RFC (Request for Comments) process -- Need approval from majority of maintainers -- Must include migration guide - -## Code of Conduct - -All participants are expected to follow our Code of Conduct. Violations can be reported to the maintainers. - -## Communication - -- **Issues**: For bug reports and feature requests -- **Discussions**: For questions and general discussion -- **Pull Requests**: For code contributions - -## Licensing - -All contributions are made under the terms of the repository's LICENSE file. -By submitting a pull request, you agree to license your contributions accordingly. - ---- - -*Last updated: 2026-07-18* diff --git a/INTEGRATION.adoc b/INTEGRATION.adoc new file mode 100644 index 00000000..a57843e3 --- /dev/null +++ b/INTEGRATION.adoc @@ -0,0 +1,305 @@ +== SPDX-License-Identifier: CC-BY-SA-4.0 + +== VeriSimDB Integration Guide + +Complete guide for the panic-attack → verisimdb-data → hypatia → +gitbot-fleet pipeline. + +=== Architecture + +.... +┌─────────────────┐ +│ Repos with │ +│ security-scan │──┐ +│ workflow │ │ +└─────────────────┘ │ + │ panic-attack scan + ↓ (JSON output) + ┌──────────────────┐ + │ verisimdb-data │ + │ (git-backed) │ + │ - scans/*.json │ + │ - index.json │ + └──────────────────┘ + │ + ↓ read scans + ┌──────────────────┐ + │ hypatia │ + │ - Logtalk rules │ + │ - Pattern detect │ + └──────────────────┘ + │ + ↓ dispatch findings + ┌──────────────────┐ + │ gitbot-fleet │ + │ - sustainabot │ + │ - echidnabot │ + │ - rhodibot │ + └──────────────────┘ +.... + +=== Current Status (2026-02-12) + +==== ✅ Complete + +* verisimdb-data repo created and deployed (GitHub + GitLab) +* Reusable scan-and-report workflow in panic-attacker repo +* 3 pilot repos with security-scan workflows (echidna, ambientops, +verisimdb) +* Hypatia VeriSimDB connector (reads scans, generates Logtalk facts) +* Hypatia pattern analyzer (summary statistics) +* Hypatia fleet dispatcher (routes findings to bots) +* Helper scripts for manual scan ingestion +* *Verified working:* 3 repos scanned, 27 weak points found, facts +generated +* *Workflow automation*: scan-and-report.yml accepts optional +VERISIMDB_PAT with GITHUB_TOKEN fallback (2026-02-12) +* *Caller workflows updated*: All 3 pilot repos pass VERISIMDB_PAT +secret to reusable workflow (2026-02-12) + +==== ⚠️ Remaining Setup + +*One manual step required*: Create a GitHub classic PAT with `+repo+` +scope and add it as `+VERISIMDB_PAT+` secret to the 3 pilot repos (see +Steps 1-2 below). The workflow files are already configured to use it. + +==== 🔲 Not Yet Implemented + +* Live GraphQL endpoints for gitbot-fleet +* Automated Logtalk rule evaluation +* Fleet dispatcher with real bot connections +* Temporal drift detection (comparing scans over time) + +=== Quick Start - Manual Workflow + +==== Scan a single repo + +[source,bash] +---- +# 1. Run scan +cd ~/Documents/hyperpolymath-repos/echidna +panic-attack assail . --output /tmp/echidna-scan.json + +# 2. Ingest result +cd ~/Documents/hyperpolymath-repos/verisimdb-data +./scripts/ingest-scan.sh echidna /tmp/echidna-scan.json + +# 3. Push to remotes +git push +git push gitlab main +---- + +==== Scan multiple repos + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/verisimdb-data +./scripts/scan-all.sh echidna ambientops verisimdb +git push +git push gitlab main +---- + +==== Test Hypatia integration + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/hypatia +mix run test_integration.exs +---- + +=== Automated Workflow Setup (PAT Method) + +==== Step 1: Create Personal Access Token + +[arabic] +. Go to https://github.com/settings/tokens +. Click "`Generate new token`" → "`Generate new token (classic)`" +. Name: `+verisimdb-dispatch+` +. Scopes: *Check `+repo+` (Full control of private repositories)* +. Generate token and *save it securely* + +==== Step 2: Add PAT to Scanning Repos + +For each repo that will send scans (echidna, ambientops, verisimdb, +etc.): + +[arabic] +. Go to repo Settings → Secrets and variables → Actions +. Click "`New repository secret`" +. Name: `+VERISIMDB_PAT+` +. Value: [paste your PAT] +. Click "`Add secret`" + +==== Step 3: Update scan-and-report Workflow ✅ DONE (2026-02-12) + +The reusable workflow +`+panic-attacker/.github/workflows/scan-and-report.yml+` now: - Accepts +optional `+VERISIMDB_PAT+` secret via `+workflow_call+` - Uses +`+${{ secrets.VERISIMDB_PAT || secrets.GITHUB_TOKEN }}+` with automatic +fallback - Uses `+curl -sf+` for silent failure detection on dispatch + +==== Step 4: Update Calling Workflows ✅ DONE (2026-02-12) + +All 3 pilot repos now pass the `+VERISIMDB_PAT+` secret: - +`+echidna/.github/workflows/security-scan.yml+` - +`+ambientops/.github/workflows/security-scan.yml+` - +`+verisimdb/.github/workflows/security-scan.yml+` + +==== Step 5: Test Automated Dispatch + +[source,bash] +---- +gh workflow run security-scan.yml --repo hyperpolymath/echidna +---- + +Then check verisimdb-data for new commits: + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/verisimdb-data +git pull +ls scans/ +cat index.json +---- + +=== Hypatia Pattern Detection + +==== Loading Scans + +[source,elixir] +---- +# In Elixir REPL +scans = Hypatia.VerisimdbConnector.fetch_all_scans() +# => Loads all scans from verisimdb-data/scans/ + +summary = Hypatia.PatternAnalyzer.generate_summary(scans) +# => %{total_repos: 3, total_weak_points: 27, ...} +---- + +==== Generating Logtalk Facts + +[source,elixir] +---- +{:ok, analysis} = Hypatia.PatternAnalyzer.analyze_all_scans() +# => Writes facts to /tmp/scan_facts.lgt +---- + +Facts format: + +[source,prolog] +---- +weak_point('echidna', 'src/rust/provers/z3.rs', 'PanicPath', 'Medium'). +weak_point('echidna', 'src/rust/ffi/mod.rs', 'UnsafeCode', 'High'). +---- + +==== Pattern Detection Rules + +See `+hypatia/prolog/pattern_detection.lgt+`: + +* `+widespread_unsafe/2+` - Find patterns appearing in 3+ repos +* `+critical_weak_points/2+` - Count critical issues per repo +* `+repo_risk_score/2+` - Calculate numeric risk score + +==== Fleet Dispatch (Placeholder) + +[source,elixir] +---- +findings = [ + %{type: :eco_score, repo: "echidna", score: 75, details: "..."}, + %{type: :proof_obligation, repo: "echidna", claim: "...", context: "..."}, + %{type: :fix_suggestion, repo: "echidna", file: "...", issue: "...", suggestion: "..."} +] + +Hypatia.PatternAnalyzer.process_findings(findings) +# => Logs dispatch to sustainabot, echidnabot, rhodibot +---- + +*Note:* GraphQL mutations are currently logged, not sent. Bots need to +expose GraphQL endpoints. + +=== Querying Scan Data + +==== Show all scan summaries + +[source,bash] +---- +jq '.repos' ~/Documents/hyperpolymath-repos/verisimdb-data/index.json +---- + +==== Find repos with most weak points + +[source,bash] +---- +jq -r '.repos | to_entries | sort_by(.value.weak_points) | reverse | map("\(.key): \(.value.weak_points)") | .[]' index.json +---- + +==== Get specific repo scan + +[source,bash] +---- +jq '.' ~/Documents/hyperpolymath-repos/verisimdb-data/scans/echidna.json | less +---- + +=== Future Enhancements + +==== Short-term + +* Create PAT and add as VERISIMDB_PAT secret (only remaining manual +step) +* Add more repos to security scanning +* Implement actual GraphQL endpoints for gitbot-fleet + +==== Medium-term + +* Temporal drift detection (compare scans over time) +* Automated rule learning (detect new patterns) +* SARIF output for GitHub Security tab +* Fly.io deployment of verisim-api (optional, beyond flat files) + +==== Long-term + +* GitHub App for organization-wide scanning +* Real-time pattern detection (immediate alerts) +* Integration with echidnabot for formal verification +* Automated fix generation via rhodibot + +=== Troubleshooting + +==== Workflow fails with "`Action not pinned to SHA`" + +*Solution:* Update to SHA-pinned version (see `+~/.claude/CLAUDE.md+` +for SHAs) + +==== Repository dispatch not received + +*Check:* 1. Is PAT configured in scanning repo secrets? 2. Does PAT have +`+repo+` scope? 3. Is reusable workflow using `+secrets.VERISIMDB_PAT+` +instead of `+GITHUB_TOKEN+`? 4. Check verisimdb-data workflow runs: +`+gh run list --repo hyperpolymath/verisimdb-data+` + +==== Scan fails with "`unexpected argument –format`" + +*Solution:* panic-attack writes JSON automatically when `+--output+` is +specified. Remove `+--format json+` flag. + +==== Hypatia can’t load scans + +*Check:* 1. Is verisimdb-data at +`+~/Documents/hyperpolymath-repos/verisimdb-data+`? 2. Do JSON files +exist in `+scans/+` directory? 3. Is Jason dependency installed? +(`+cd hypatia && mix deps.get+`) + +=== Support + +* panic-attack issues: +https://github.com/hyperpolymath/panic-attacker/issues +* verisimdb issues: https://github.com/hyperpolymath/verisimdb/issues +* hypatia issues: https://github.com/hyperpolymath/hypatia/issues + +=== References + +* SONNET-TASKS.md - Original implementation tasks +* scripts/README.md - Helper script documentation +* hypatia/test_integration.exs - Integration test example +* panic-attacker/.claude/CLAUDE.md - panic-attack documentation +* verisimdb/.claude/CLAUDE.md - VeriSimDB architecture diff --git a/INTEGRATION.md b/INTEGRATION.md deleted file mode 100644 index 4c99aa31..00000000 --- a/INTEGRATION.md +++ /dev/null @@ -1,274 +0,0 @@ -# SPDX-License-Identifier: CC-BY-SA-4.0 - -# VeriSimDB Integration Guide - -Complete guide for the panic-attack → verisimdb-data → hypatia → gitbot-fleet pipeline. - -## Architecture - -``` -┌─────────────────┐ -│ Repos with │ -│ security-scan │──┐ -│ workflow │ │ -└─────────────────┘ │ - │ panic-attack scan - ↓ (JSON output) - ┌──────────────────┐ - │ verisimdb-data │ - │ (git-backed) │ - │ - scans/*.json │ - │ - index.json │ - └──────────────────┘ - │ - ↓ read scans - ┌──────────────────┐ - │ hypatia │ - │ - Logtalk rules │ - │ - Pattern detect │ - └──────────────────┘ - │ - ↓ dispatch findings - ┌──────────────────┐ - │ gitbot-fleet │ - │ - sustainabot │ - │ - echidnabot │ - │ - rhodibot │ - └──────────────────┘ -``` - -## Current Status (2026-02-12) - -### ✅ Complete - -- verisimdb-data repo created and deployed (GitHub + GitLab) -- Reusable scan-and-report workflow in panic-attacker repo -- 3 pilot repos with security-scan workflows (echidna, ambientops, verisimdb) -- Hypatia VeriSimDB connector (reads scans, generates Logtalk facts) -- Hypatia pattern analyzer (summary statistics) -- Hypatia fleet dispatcher (routes findings to bots) -- Helper scripts for manual scan ingestion -- **Verified working:** 3 repos scanned, 27 weak points found, facts generated -- **Workflow automation**: scan-and-report.yml accepts optional VERISIMDB_PAT with GITHUB_TOKEN fallback (2026-02-12) -- **Caller workflows updated**: All 3 pilot repos pass VERISIMDB_PAT secret to reusable workflow (2026-02-12) - -### ⚠️ Remaining Setup - -**One manual step required**: Create a GitHub classic PAT with `repo` scope and add it as `VERISIMDB_PAT` secret to the 3 pilot repos (see Steps 1-2 below). The workflow files are already configured to use it. - -### 🔲 Not Yet Implemented - -- Live GraphQL endpoints for gitbot-fleet -- Automated Logtalk rule evaluation -- Fleet dispatcher with real bot connections -- Temporal drift detection (comparing scans over time) - -## Quick Start - Manual Workflow - -### Scan a single repo - -```bash -# 1. Run scan -cd ~/Documents/hyperpolymath-repos/echidna -panic-attack assail . --output /tmp/echidna-scan.json - -# 2. Ingest result -cd ~/Documents/hyperpolymath-repos/verisimdb-data -./scripts/ingest-scan.sh echidna /tmp/echidna-scan.json - -# 3. Push to remotes -git push -git push gitlab main -``` - -### Scan multiple repos - -```bash -cd ~/Documents/hyperpolymath-repos/verisimdb-data -./scripts/scan-all.sh echidna ambientops verisimdb -git push -git push gitlab main -``` - -### Test Hypatia integration - -```bash -cd ~/Documents/hyperpolymath-repos/hypatia -mix run test_integration.exs -``` - -## Automated Workflow Setup (PAT Method) - -### Step 1: Create Personal Access Token - -1. Go to https://github.com/settings/tokens -2. Click "Generate new token" → "Generate new token (classic)" -3. Name: `verisimdb-dispatch` -4. Scopes: **Check `repo` (Full control of private repositories)** -5. Generate token and **save it securely** - -### Step 2: Add PAT to Scanning Repos - -For each repo that will send scans (echidna, ambientops, verisimdb, etc.): - -1. Go to repo Settings → Secrets and variables → Actions -2. Click "New repository secret" -3. Name: `VERISIMDB_PAT` -4. Value: [paste your PAT] -5. Click "Add secret" - -### Step 3: Update scan-and-report Workflow ✅ DONE (2026-02-12) - -The reusable workflow `panic-attacker/.github/workflows/scan-and-report.yml` now: -- Accepts optional `VERISIMDB_PAT` secret via `workflow_call` -- Uses `${{ secrets.VERISIMDB_PAT || secrets.GITHUB_TOKEN }}` with automatic fallback -- Uses `curl -sf` for silent failure detection on dispatch - -### Step 4: Update Calling Workflows ✅ DONE (2026-02-12) - -All 3 pilot repos now pass the `VERISIMDB_PAT` secret: -- `echidna/.github/workflows/security-scan.yml` -- `ambientops/.github/workflows/security-scan.yml` -- `verisimdb/.github/workflows/security-scan.yml` - -### Step 5: Test Automated Dispatch - -```bash -gh workflow run security-scan.yml --repo hyperpolymath/echidna -``` - -Then check verisimdb-data for new commits: - -```bash -cd ~/Documents/hyperpolymath-repos/verisimdb-data -git pull -ls scans/ -cat index.json -``` - -## Hypatia Pattern Detection - -### Loading Scans - -```elixir -# In Elixir REPL -scans = Hypatia.VerisimdbConnector.fetch_all_scans() -# => Loads all scans from verisimdb-data/scans/ - -summary = Hypatia.PatternAnalyzer.generate_summary(scans) -# => %{total_repos: 3, total_weak_points: 27, ...} -``` - -### Generating Logtalk Facts - -```elixir -{:ok, analysis} = Hypatia.PatternAnalyzer.analyze_all_scans() -# => Writes facts to /tmp/scan_facts.lgt -``` - -Facts format: -```prolog -weak_point('echidna', 'src/rust/provers/z3.rs', 'PanicPath', 'Medium'). -weak_point('echidna', 'src/rust/ffi/mod.rs', 'UnsafeCode', 'High'). -``` - -### Pattern Detection Rules - -See `hypatia/prolog/pattern_detection.lgt`: - -- `widespread_unsafe/2` - Find patterns appearing in 3+ repos -- `critical_weak_points/2` - Count critical issues per repo -- `repo_risk_score/2` - Calculate numeric risk score - -### Fleet Dispatch (Placeholder) - -```elixir -findings = [ - %{type: :eco_score, repo: "echidna", score: 75, details: "..."}, - %{type: :proof_obligation, repo: "echidna", claim: "...", context: "..."}, - %{type: :fix_suggestion, repo: "echidna", file: "...", issue: "...", suggestion: "..."} -] - -Hypatia.PatternAnalyzer.process_findings(findings) -# => Logs dispatch to sustainabot, echidnabot, rhodibot -``` - -**Note:** GraphQL mutations are currently logged, not sent. Bots need to expose GraphQL endpoints. - -## Querying Scan Data - -### Show all scan summaries - -```bash -jq '.repos' ~/Documents/hyperpolymath-repos/verisimdb-data/index.json -``` - -### Find repos with most weak points - -```bash -jq -r '.repos | to_entries | sort_by(.value.weak_points) | reverse | map("\(.key): \(.value.weak_points)") | .[]' index.json -``` - -### Get specific repo scan - -```bash -jq '.' ~/Documents/hyperpolymath-repos/verisimdb-data/scans/echidna.json | less -``` - -## Future Enhancements - -### Short-term -- Create PAT and add as VERISIMDB_PAT secret (only remaining manual step) -- Add more repos to security scanning -- Implement actual GraphQL endpoints for gitbot-fleet - -### Medium-term -- Temporal drift detection (compare scans over time) -- Automated rule learning (detect new patterns) -- SARIF output for GitHub Security tab -- Fly.io deployment of verisim-api (optional, beyond flat files) - -### Long-term -- GitHub App for organization-wide scanning -- Real-time pattern detection (immediate alerts) -- Integration with echidnabot for formal verification -- Automated fix generation via rhodibot - -## Troubleshooting - -### Workflow fails with "Action not pinned to SHA" - -**Solution:** Update to SHA-pinned version (see `~/.claude/CLAUDE.md` for SHAs) - -### Repository dispatch not received - -**Check:** -1. Is PAT configured in scanning repo secrets? -2. Does PAT have `repo` scope? -3. Is reusable workflow using `secrets.VERISIMDB_PAT` instead of `GITHUB_TOKEN`? -4. Check verisimdb-data workflow runs: `gh run list --repo hyperpolymath/verisimdb-data` - -### Scan fails with "unexpected argument --format" - -**Solution:** panic-attack writes JSON automatically when `--output` is specified. Remove `--format json` flag. - -### Hypatia can't load scans - -**Check:** -1. Is verisimdb-data at `~/Documents/hyperpolymath-repos/verisimdb-data`? -2. Do JSON files exist in `scans/` directory? -3. Is Jason dependency installed? (`cd hypatia && mix deps.get`) - -## Support - -- panic-attack issues: https://github.com/hyperpolymath/panic-attacker/issues -- verisimdb issues: https://github.com/hyperpolymath/verisimdb/issues -- hypatia issues: https://github.com/hyperpolymath/hypatia/issues - -## References - -- SONNET-TASKS.md - Original implementation tasks -- scripts/README.md - Helper script documentation -- hypatia/test_integration.exs - Integration test example -- panic-attacker/.claude/CLAUDE.md - panic-attack documentation -- verisimdb/.claude/CLAUDE.md - VeriSimDB architecture diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc new file mode 100644 index 00000000..7d5132fb --- /dev/null +++ b/PROOF-NEEDS.adoc @@ -0,0 +1,12 @@ +== PROOF-NEEDS.md + +=== Template ABI Cleanup (2026-03-29) + +Template ABI removed – was creating false impression of formal +verification. The removed files (Types.idr, Layout.idr, Foreign.idr) +contained only RSR template scaffolding with unresolved +\{\{PROJECT}}/\{\{AUTHOR}} placeholders and no domain-specific proofs. + +When this project needs formal ABI verification, create domain-specific +Idris2 proofs following the pattern in repos like `+typed-wasm+`, +`+proven+`, `+echidna+`, or `+boj-server+`. diff --git a/PROOF-NEEDS.md b/PROOF-NEEDS.md deleted file mode 100644 index 89503202..00000000 --- a/PROOF-NEEDS.md +++ /dev/null @@ -1,10 +0,0 @@ -# PROOF-NEEDS.md - -## Template ABI Cleanup (2026-03-29) - -Template ABI removed -- was creating false impression of formal verification. -The removed files (Types.idr, Layout.idr, Foreign.idr) contained only RSR template -scaffolding with unresolved {{PROJECT}}/{{AUTHOR}} placeholders and no domain-specific proofs. - -When this project needs formal ABI verification, create domain-specific Idris2 proofs -following the pattern in repos like `typed-wasm`, `proven`, `echidna`, or `boj-server`. diff --git a/SCAN-REPORT-2026-02-08.adoc b/SCAN-REPORT-2026-02-08.adoc new file mode 100644 index 00000000..4fa14671 --- /dev/null +++ b/SCAN-REPORT-2026-02-08.adoc @@ -0,0 +1,232 @@ +== VeriSimDB Scan Loading Report + +*Date:* 2026-02-08 *Status:* ✅ Complete + +=== Summary + +Successfully loaded panic-attack scan results into verisimdb-data for 15 +hyperpolymath repositories. All data has been committed and pushed to +both GitHub and GitLab. Hypatia integration verified and working. + +=== Scan Results + +==== Total Statistics + +* *Total repos scanned:* 15 +* *Total weak points found:* 67 +* *Average weak points per repo:* 4.47 +* *Repos with zero weak points:* 5 (33%) + +==== Repos by Weak Point Count (Descending) + +[cols=",,,",options="header",] +|=== +|Rank |Repository |Weak Points |Status +|1 |echidna |15 |🔴 High priority +|2 |verisimdb |12 |🔴 High priority +|3 |my-lang |11 |🔴 High priority +|4 |hypatia |7 |🟡 Medium priority +|5 |panic-attacker |5 |🟡 Medium priority +|6 |gitbot-fleet |5 |🟡 Medium priority +|7 |robot-repo-automaton |4 |🟡 Medium priority +|8 |affinescript |4 |🟡 Medium priority +|9 |lithoglyph |3 |🟢 Low priority +|10 |oblibeny |1 |🟢 Low priority +|11 |consent-aware-http |0 |✅ Clean +|12 |palimpsest-license |0 |✅ Clean +|13 |a2ml |0 |✅ Clean +|14 |http-capability-gateway |0 |✅ Clean +|15 |ambientops |0 |✅ Clean +|=== + +==== Repos Not Scanned + +*scaffoldia* - Could not scan (panic-attack doesn’t support Haskell yet) +*rsr-template-repo* - Scan failed (likely template structure issue) + +=== Integration Pipeline Status + +==== ✅ Working Components + +[arabic] +. *panic-attack scanner* - Successfully scans Rust codebases +. *verisimdb-data repository* - Stores all scan results in JSON format +. *Helper scripts* - `+ingest-scan.sh+` and `+scan-all.sh+` working +perfectly +. *Hypatia connector* - Successfully reads all 15 scans +. *Logtalk fact generation* - 67 weak_point facts generated to +`+/tmp/scan_facts.lgt+` +. *Pattern analyzer* - Summary statistics generated correctly +. *Git synchronization* - All changes pushed to GitHub and GitLab + +==== Data Flow Verification + +.... +panic-attack scan → JSON files → verisimdb-data/scans/ + ↓ + index.json updated + ↓ + Git commit/push + ↓ + Hypatia reads via connector + ↓ + Logtalk facts generated + ↓ + Pattern analysis complete +.... + +=== Weak Point Analysis + +==== By Type (from echidna sample) + +* *PanicPath:* 12 occurrences (80%) +* *UnsafeCode:* 3 occurrences (20%) + +==== By Severity (from echidna sample) + +* *High:* 3 weak points (unsafe FFI code) +* *Medium:* 12 weak points (panic paths in tests/provers) + +==== Common Patterns + +[arabic] +. *Test files with panics:* Most repos have panic paths in test code +. *FFI boundaries:* Unsafe code at Rust FFI interfaces +. *Prover integration points:* Multiple prover backends with panic +handling + +=== File Locations + +==== Scan Data + +.... +~/Documents/hyperpolymath-repos/verisimdb-data/ +├── scans/ +│ ├── echidna.json (19K) +│ ├── verisimdb.json (8.4K) +│ ├── my-lang.json (19K) +│ ├── hypatia.json (11K) +│ ├── affinescript.json (6.1K) +│ ├── gitbot-fleet.json (4.8K) +│ ├── lithoglyph.json (6.0K) +│ ├── panic-attacker.json (4.2K) +│ ├── robot-repo-automaton.json (4.2K) +│ ├── oblibeny.json (1.6K) +│ ├── http-capability-gateway.json (1.4K) +│ ├── ambientops.json (1.4K) +│ ├── consent-aware-http.json (654B) +│ ├── palimpsest-license.json (848B) +│ └── a2ml.json (359B) +└── index.json (master index with metadata) +.... + +==== Hypatia Integration + +.... +~/Documents/hyperpolymath-repos/hypatia/ +├── lib/hypatia/verisimdb_connector.ex (reads scans) +├── lib/hypatia/pattern_analyzer.ex (generates facts) +└── test_integration.exs (integration test) + +/tmp/scan_facts.lgt (67 Logtalk facts, 138 lines) +.... + +=== Git History + +==== Commits Created + +.... +489ec0a scan: update panic-attacker results +b02e060 scan: update robot-repo-automaton results +f78e05b scan: update hypatia results +54b64dd scan: update my-lang results +2106f3b scan: update lithoglyph results +7a087f0 scan: update consent-aware-http results +5d8f979 scan: update oblibeny results +1d6600a scan: update palimpsest-license results +3ecefc2 scan: update gitbot-fleet results +1e1d207 scan: update a2ml results +5d2545b scan: update affinescript results +c3ae834 scan: update http-capability-gateway results +.... + +==== Remote Status + +* *GitHub:* All commits pushed to `+origin/main+` +* *GitLab:* All commits mirrored to `+gitlab/main+` + +=== Next Steps + +==== Immediate + +[arabic] +. ✅ Complete - All scan results loaded +. ✅ Complete - Data pushed to remotes +. ✅ Complete - Hypatia integration verified + +==== Short-term + +[arabic] +. *Address high-priority repos* - echidna, verisimdb, my-lang +. *Set up automated scanning* - Add security-scan workflows to scanned +repos +. *Enable PAT-based dispatch* - Allow automated scan uploads + +==== Medium-term + +[arabic] +. *Expand coverage* - Scan remaining hyperpolymath repos +. *Add Haskell support to panic-attack* - Enable scaffoldia scanning +. *Implement gitbot-fleet GraphQL endpoints* - Connect bots to findings + +==== Long-term + +[arabic] +. *Temporal drift detection* - Track weak points over time +. *Automated fix generation* - rhodibot integration +. *Formal verification triggers* - echidnabot integration + +=== Testing Commands + +==== Re-run Hypatia integration test + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/hypatia +mix run test_integration.exs +---- + +==== View scan results + +[source,bash] +---- +# Summary +jq '.repos' ~/Documents/hyperpolymath-repos/verisimdb-data/index.json + +# Specific repo +jq '.' ~/Documents/hyperpolymath-repos/verisimdb-data/scans/echidna.json + +# Sorted by weak points +jq -r '.repos | to_entries | sort_by(.value.weak_points) | reverse | map("\(.key): \(.value.weak_points) weak points") | .[]' ~/Documents/hyperpolymath-repos/verisimdb-data/index.json +---- + +==== View Logtalk facts + +[source,bash] +---- +cat /tmp/scan_facts.lgt | less +---- + +=== References + +* *Integration Guide:* +`+~/Documents/hyperpolymath-repos/verisimdb-data/INTEGRATION.md+` +* *Helper Scripts:* +`+~/Documents/hyperpolymath-repos/verisimdb-data/scripts/+` +* *panic-attack Tool:* +`+/var$REPOS_DIR/panic-attacker/target/release/panic-attack+` + +''''' + +*Task Complete:* VeriSimDB scan loading pipeline fully operational with +15 repos scanned and Hypatia integration verified. diff --git a/SCAN-REPORT-2026-02-08.md b/SCAN-REPORT-2026-02-08.md deleted file mode 100644 index 4a8f2363..00000000 --- a/SCAN-REPORT-2026-02-08.md +++ /dev/null @@ -1,196 +0,0 @@ -# VeriSimDB Scan Loading Report -**Date:** 2026-02-08 -**Status:** ✅ Complete - -## Summary - -Successfully loaded panic-attack scan results into verisimdb-data for 15 hyperpolymath repositories. All data has been committed and pushed to both GitHub and GitLab. Hypatia integration verified and working. - -## Scan Results - -### Total Statistics -- **Total repos scanned:** 15 -- **Total weak points found:** 67 -- **Average weak points per repo:** 4.47 -- **Repos with zero weak points:** 5 (33%) - -### Repos by Weak Point Count (Descending) - -| Rank | Repository | Weak Points | Status | -|------|-----------|------------|--------| -| 1 | echidna | 15 | 🔴 High priority | -| 2 | verisimdb | 12 | 🔴 High priority | -| 3 | my-lang | 11 | 🔴 High priority | -| 4 | hypatia | 7 | 🟡 Medium priority | -| 5 | panic-attacker | 5 | 🟡 Medium priority | -| 6 | gitbot-fleet | 5 | 🟡 Medium priority | -| 7 | robot-repo-automaton | 4 | 🟡 Medium priority | -| 8 | affinescript | 4 | 🟡 Medium priority | -| 9 | lithoglyph | 3 | 🟢 Low priority | -| 10 | oblibeny | 1 | 🟢 Low priority | -| 11 | consent-aware-http | 0 | ✅ Clean | -| 12 | palimpsest-license | 0 | ✅ Clean | -| 13 | a2ml | 0 | ✅ Clean | -| 14 | http-capability-gateway | 0 | ✅ Clean | -| 15 | ambientops | 0 | ✅ Clean | - -### Repos Not Scanned - -**scaffoldia** - Could not scan (panic-attack doesn't support Haskell yet) -**rsr-template-repo** - Scan failed (likely template structure issue) - -## Integration Pipeline Status - -### ✅ Working Components - -1. **panic-attack scanner** - Successfully scans Rust codebases -2. **verisimdb-data repository** - Stores all scan results in JSON format -3. **Helper scripts** - `ingest-scan.sh` and `scan-all.sh` working perfectly -4. **Hypatia connector** - Successfully reads all 15 scans -5. **Logtalk fact generation** - 67 weak_point facts generated to `/tmp/scan_facts.lgt` -6. **Pattern analyzer** - Summary statistics generated correctly -7. **Git synchronization** - All changes pushed to GitHub and GitLab - -### Data Flow Verification - -``` -panic-attack scan → JSON files → verisimdb-data/scans/ - ↓ - index.json updated - ↓ - Git commit/push - ↓ - Hypatia reads via connector - ↓ - Logtalk facts generated - ↓ - Pattern analysis complete -``` - -## Weak Point Analysis - -### By Type (from echidna sample) -- **PanicPath:** 12 occurrences (80%) -- **UnsafeCode:** 3 occurrences (20%) - -### By Severity (from echidna sample) -- **High:** 3 weak points (unsafe FFI code) -- **Medium:** 12 weak points (panic paths in tests/provers) - -### Common Patterns -1. **Test files with panics:** Most repos have panic paths in test code -2. **FFI boundaries:** Unsafe code at Rust FFI interfaces -3. **Prover integration points:** Multiple prover backends with panic handling - -## File Locations - -### Scan Data -``` -~/Documents/hyperpolymath-repos/verisimdb-data/ -├── scans/ -│ ├── echidna.json (19K) -│ ├── verisimdb.json (8.4K) -│ ├── my-lang.json (19K) -│ ├── hypatia.json (11K) -│ ├── affinescript.json (6.1K) -│ ├── gitbot-fleet.json (4.8K) -│ ├── lithoglyph.json (6.0K) -│ ├── panic-attacker.json (4.2K) -│ ├── robot-repo-automaton.json (4.2K) -│ ├── oblibeny.json (1.6K) -│ ├── http-capability-gateway.json (1.4K) -│ ├── ambientops.json (1.4K) -│ ├── consent-aware-http.json (654B) -│ ├── palimpsest-license.json (848B) -│ └── a2ml.json (359B) -└── index.json (master index with metadata) -``` - -### Hypatia Integration -``` -~/Documents/hyperpolymath-repos/hypatia/ -├── lib/hypatia/verisimdb_connector.ex (reads scans) -├── lib/hypatia/pattern_analyzer.ex (generates facts) -└── test_integration.exs (integration test) - -/tmp/scan_facts.lgt (67 Logtalk facts, 138 lines) -``` - -## Git History - -### Commits Created -``` -489ec0a scan: update panic-attacker results -b02e060 scan: update robot-repo-automaton results -f78e05b scan: update hypatia results -54b64dd scan: update my-lang results -2106f3b scan: update lithoglyph results -7a087f0 scan: update consent-aware-http results -5d8f979 scan: update oblibeny results -1d6600a scan: update palimpsest-license results -3ecefc2 scan: update gitbot-fleet results -1e1d207 scan: update a2ml results -5d2545b scan: update affinescript results -c3ae834 scan: update http-capability-gateway results -``` - -### Remote Status -- **GitHub:** All commits pushed to `origin/main` -- **GitLab:** All commits mirrored to `gitlab/main` - -## Next Steps - -### Immediate -1. ✅ Complete - All scan results loaded -2. ✅ Complete - Data pushed to remotes -3. ✅ Complete - Hypatia integration verified - -### Short-term -1. **Address high-priority repos** - echidna, verisimdb, my-lang -2. **Set up automated scanning** - Add security-scan workflows to scanned repos -3. **Enable PAT-based dispatch** - Allow automated scan uploads - -### Medium-term -1. **Expand coverage** - Scan remaining hyperpolymath repos -2. **Add Haskell support to panic-attack** - Enable scaffoldia scanning -3. **Implement gitbot-fleet GraphQL endpoints** - Connect bots to findings - -### Long-term -1. **Temporal drift detection** - Track weak points over time -2. **Automated fix generation** - rhodibot integration -3. **Formal verification triggers** - echidnabot integration - -## Testing Commands - -### Re-run Hypatia integration test -```bash -cd ~/Documents/hyperpolymath-repos/hypatia -mix run test_integration.exs -``` - -### View scan results -```bash -# Summary -jq '.repos' ~/Documents/hyperpolymath-repos/verisimdb-data/index.json - -# Specific repo -jq '.' ~/Documents/hyperpolymath-repos/verisimdb-data/scans/echidna.json - -# Sorted by weak points -jq -r '.repos | to_entries | sort_by(.value.weak_points) | reverse | map("\(.key): \(.value.weak_points) weak points") | .[]' ~/Documents/hyperpolymath-repos/verisimdb-data/index.json -``` - -### View Logtalk facts -```bash -cat /tmp/scan_facts.lgt | less -``` - -## References - -- **Integration Guide:** `~/Documents/hyperpolymath-repos/verisimdb-data/INTEGRATION.md` -- **Helper Scripts:** `~/Documents/hyperpolymath-repos/verisimdb-data/scripts/` -- **panic-attack Tool:** `/var$REPOS_DIR/panic-attacker/target/release/panic-attack` - ---- - -**Task Complete:** VeriSimDB scan loading pipeline fully operational with 15 repos scanned and Hypatia integration verified. diff --git a/SECURITY.adoc b/SECURITY.adoc new file mode 100644 index 00000000..bbb8bab1 --- /dev/null +++ b/SECURITY.adoc @@ -0,0 +1,21 @@ +== Security Policy + +=== Reporting a Vulnerability + +Please report security vulnerabilities to: j.d.a.jewell@open.ac.uk + +Do NOT open a public issue for security vulnerabilities. + +=== Supported Versions + +[cols=",",options="header",] +|=== +|Version |Supported +|Latest |Yes +|=== + +=== Security Practices + +* All dependencies are monitored via Dependabot +* CodeQL analysis runs on every push +* SPDX-License-Identifier: CC-BY-SA-4.0 diff --git a/SECURITY.md b/SECURITY.md deleted file mode 100644 index d91da4e6..00000000 --- a/SECURITY.md +++ /dev/null @@ -1,19 +0,0 @@ -# Security Policy - -## Reporting a Vulnerability - -Please report security vulnerabilities to: j.d.a.jewell@open.ac.uk - -Do NOT open a public issue for security vulnerabilities. - -## Supported Versions - -| Version | Supported | -|---------|-----------| -| Latest | Yes | - -## Security Practices - -- All dependencies are monitored via Dependabot -- CodeQL analysis runs on every push -- SPDX-License-Identifier: CC-BY-SA-4.0 diff --git a/SESSION-SUMMARY.adoc b/SESSION-SUMMARY.adoc new file mode 100644 index 00000000..9797ef23 --- /dev/null +++ b/SESSION-SUMMARY.adoc @@ -0,0 +1,331 @@ +== SPDX-License-Identifier: CC-BY-SA-4.0 + +== Session Summary - 2026-02-08 + +=== Complete Integration Pipeline Delivered + +All 5 tasks from `+verisimdb/SONNET-TASKS.md+` completed + comprehensive +testing and documentation. + +''''' + +=== ✅ Completed Work + +==== 1. verisimdb-data Repository (Task 1) + +*Status:* DEPLOYED ✓ + +* Created git-backed flat-file storage repo +* Directory structure: `+scans/+`, `+hardware/+`, `+drift/+`, +`+index.json+` +* Ingest workflow: `+.github/workflows/ingest.yml+` (accepts +repository_dispatch events) +* *Deployed to:* +** GitHub: https://github.com/hyperpolymath/verisimdb-data +** GitLab: https://gitlab.com/hyperpolymath/verisimdb-data + +*Fixes applied:* - SHA-pinned all GitHub Actions - Fixed multi-line +commit message formatting (heredoc) + +==== 2. Reusable Scan Workflow (Task 2) + +*Status:* WORKING ✓ + +* File: `+panic-attacker/.github/workflows/scan-and-report.yml+` +* Other repos can call: +`+uses: hyperpolymath/panic-attacker/.github/workflows/scan-and-report.yml@main+` + +*Fixes applied:* - SHA-pinned `+dtolnay/rust-toolchain+` action - +Removed non-existent `+--format json+` flag (panic-attack writes JSON +automatically with `+--output+`) + +==== 3. Pilot Repo Deployment (Task 3) + +*Status:* DEPLOYED ✓ + +All 3 pilot repos have `+security-scan.yml+` workflows: - ✓ echidna - ✓ +ambientops - ✓ verisimdb + +*Scan results:* - echidna: 15 weak points (13 Medium, 2 High) - +ambientops: 0 weak points (CLEAN!) - verisimdb: 12 weak points + +==== 4. Hypatia VeriSimDB Connector (Task 4) + +*Status:* WORKING ✓ + +*Files created:* - `+lib/verisimdb_connector.ex+` - Reads scans from +verisimdb-data, transforms to Logtalk facts - +`+lib/pattern_analyzer.ex+` - Analyzes scans, generates summaries - +`+prolog/pattern_detection.lgt+` - Logtalk rules for pattern detection - +`+mix.exs+` - Project file with Jason dependency - +`+test_integration.exs+` - Integration test script + +*Verified working:* - Loaded all 3 scans from verisimdb-data - Generated +summary: 27 total weak points across 3 repos - Created Logtalk facts +file: `+/tmp/scan_facts.lgt+` + +*Fixes applied:* - Fixed field name mapping: panic-attack uses +`+"location"+` not `+"file"+` + +==== 5. Fleet Dispatcher (Task 5) + +*Status:* INITIAL ✓ + +*Files created:* - `+lib/fleet_dispatcher.ex+` - Routes findings to +sustainabot, echidnabot, rhodibot - GraphQL mutations defined for each +bot type + +*Current behavior:* - Findings are logged (not yet sent to live bots) - +GraphQL endpoints need to be implemented by bot repos + +''''' + +=== 📊 Real Data Results + +==== Scans Completed + +[cols=",,,,,",options="header",] +|=== +|Repo |Weak Points |Critical |High |Medium |Low +|echidna |15 |0 |3 |12 |0 +|ambientops |0 |0 |0 |0 |0 +|verisimdb |12 |0 |0 |12 |0 +|*Total* |*27* |*0* |*3* |*24* |*0* +|=== + +==== Pattern Detection + +Generated 27 Logtalk facts: + +[source,prolog] +---- +weak_point('echidna', 'src/rust/ffi/mod.rs', 'UnsafeCode', 'High'). +weak_point('echidna', 'src/rust/provers/z3.rs', 'PanicPath', 'Medium'). +weak_point('verisimdb', 'rust-core/verisim-graph/src/lib.rs', 'PanicPath', 'Medium'). +... +---- + +==== Notable Findings + +*echidna high-severity issues:* - 7 unsafe blocks in +`+src/rust/ffi/mod.rs+` (FFI boundary) - 7 unsafe blocks in +`+src/rust/proof_search.rs+` (optimization) - 1 unsafe block in HOL +tree-sitter bindings (expected) + +*ambientops:* Clean! Zero weak points. + +*verisimdb:* 12 unwrap/expect calls in modality stores (moderate risk) + +''''' + +=== 🛠️ Helper Tools Created + +==== Manual Scan Ingestion + +*scripts/ingest-scan.sh* + +[source,bash] +---- +./scripts/ingest-scan.sh echidna /tmp/echidna-scan.json +---- + +*scripts/scan-all.sh* + +[source,bash] +---- +./scripts/scan-all.sh # scans echidna, ambientops, verisimdb +---- + +==== Integration Testing + +*hypatia/test_integration.exs* + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/hypatia +mix run test_integration.exs +---- + +''''' + +=== 📚 Documentation Created + +==== INTEGRATION.md (317 lines) + +Complete guide covering: - Architecture diagram - Current status and +limitations - Quick start (manual workflow) - PAT setup instructions +(for automated dispatch) - Hypatia pattern detection examples - Querying +scan data - Troubleshooting guide + +==== scripts/README.md + +Documentation for helper scripts with examples. + +''''' + +=== ⚠️ Known Limitation + +==== Automated Dispatch Blocked + +*Issue:* `+GITHUB_TOKEN+` cannot trigger `+repository_dispatch+` events +in other repos (GitHub security policy). + +*Current workaround:* Manual scan ingestion using helper scripts. + +*Solution (for automated workflow):* + +[arabic] +. Create Personal Access Token (PAT) with `+repo+` scope +. Add as secret `+VERISIMDB_PAT+` in scanning repos +. Update `+scan-and-report.yml+` to use PAT instead of GITHUB_TOKEN +. See `+INTEGRATION.md+` for complete setup instructions + +''''' + +=== 📈 State Updates + +==== verisimdb STATE.scm + +Updated with: - GitHub CI integration: *COMPLETE (100%)* - Hypatia +pipeline: *INITIAL (40%)* - Session history entry for 2026-02-08 + +==== Repositories Updated + +*Commits & Pushes:* - verisimdb-data: 5 commits (GitHub + GitLab) - +panic-attacker: 3 commits (GitHub) - echidna: 1 commit (GitHub) - +ambientops: 1 commit (GitHub) - verisimdb: 2 commits (GitHub) - hypatia: +2 commits (GitHub) + +*Total:* 14 commits across 6 repos + +''''' + +=== 🎯 Next Steps (for Sonnet or future work) + +==== Immediate + +[arabic] +. Configure PAT for automated dispatch (see INTEGRATION.md) +. Add more repos to security scanning +. Monitor weekly scans (scheduled via cron) + +==== Short-term + +[arabic, start=4] +. Implement GraphQL endpoints in gitbot-fleet bots +. Wire up live fleet dispatcher +. Add temporal drift detection (compare scans over time) + +==== Medium-term + +[arabic, start=7] +. SARIF output for GitHub Security tab integration +. Automated rule learning (detect new patterns) +. Real-time pattern alerts + +==== Long-term + +[arabic, start=10] +. GitHub App for organization-wide scanning +. Integration with echidnabot for formal verification +. Automated fix generation via rhodibot + +''''' + +=== 📋 Quick Reference + +==== Scan a repo manually + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/echidna +panic-attack assail . --output /tmp/echidna-scan.json +cd ~/Documents/hyperpolymath-repos/verisimdb-data +./scripts/ingest-scan.sh echidna /tmp/echidna-scan.json +git push && git push gitlab main +---- + +==== Test Hypatia integration + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/hypatia +mix run test_integration.exs +---- + +==== View scan results + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/verisimdb-data +jq '.repos' index.json +cat scans/echidna.json | less +---- + +==== Trigger automated scan (requires PAT) + +[source,bash] +---- +gh workflow run security-scan.yml --repo hyperpolymath/echidna +---- + +''''' + +=== ✨ Summary + +*Complete integration pipeline delivered:* - ✅ Git-backed data storage +(verisimdb-data) - ✅ Reusable scan workflow (panic-attacker) - ✅ 3 +pilot repos scanning (echidna, ambientops, verisimdb) - ✅ Pattern +detection (hypatia) - ✅ Fleet dispatcher (hypatia) - ✅ Helper scripts +for manual workflow - ✅ Comprehensive documentation - ✅ Verified with +real data (27 weak points found) + +*Manual workflow fully operational.* Automated workflow requires PAT +configuration (15 minutes of setup). + +*All code committed and pushed to GitHub + GitLab.* + +''''' + +''''' + +== Addendum: 2026-02-12 - Workflow Automation Update + +=== Changes Made + +==== Workflow File Updates (Steps 3-4 of PAT Setup) + +All workflow files have been updated to support automated cross-repo +dispatch: + +[arabic] +. *panic-attacker/scan-and-report.yml* (reusable workflow): +* Added `+secrets:+` block accepting optional `+VERISIMDB_PAT+` +* Dispatch token now uses +`+${{ secrets.VERISIMDB_PAT || secrets.GITHUB_TOKEN }}+` fallback +* Added `+-sf+` flag to curl for silent failure detection +. *Caller workflows* (all 3 pilot repos updated): +* `+echidna/.github/workflows/security-scan.yml+` — passes VERISIMDB_PAT +* `+ambientops/.github/workflows/security-scan.yml+` — passes +VERISIMDB_PAT +* `+verisimdb/.github/workflows/security-scan.yml+` — passes +VERISIMDB_PAT + +==== Git Mirroring Fixes + +* *ambientops*: Added GitLab remote, unshallowed clone, pushed +successfully +* *echidna GitLab*: Diverged history (protected branch + expired PAT +blocks force push) + +==== Remaining Manual Step + +Only Steps 1-2 of the PAT setup remain: 1. Create classic PAT at +https://github.com/settings/tokens with `+repo+` scope 2. Add as +`+VERISIMDB_PAT+` secret to echidna, ambientops, and verisimdb repos + +Once done, automated scanning will work end-to-end. + +''''' + +Generated: 2026-02-12T18:30:00Z diff --git a/SESSION-SUMMARY.md b/SESSION-SUMMARY.md deleted file mode 100644 index 66636ffd..00000000 --- a/SESSION-SUMMARY.md +++ /dev/null @@ -1,311 +0,0 @@ -# SPDX-License-Identifier: CC-BY-SA-4.0 - -# Session Summary - 2026-02-08 - -## Complete Integration Pipeline Delivered - -All 5 tasks from `verisimdb/SONNET-TASKS.md` completed + comprehensive testing and documentation. - ---- - -## ✅ Completed Work - -### 1. verisimdb-data Repository (Task 1) - -**Status:** DEPLOYED ✓ - -- Created git-backed flat-file storage repo -- Directory structure: `scans/`, `hardware/`, `drift/`, `index.json` -- Ingest workflow: `.github/workflows/ingest.yml` (accepts repository_dispatch events) -- **Deployed to:** - - GitHub: https://github.com/hyperpolymath/verisimdb-data - - GitLab: https://gitlab.com/hyperpolymath/verisimdb-data - -**Fixes applied:** -- SHA-pinned all GitHub Actions -- Fixed multi-line commit message formatting (heredoc) - -### 2. Reusable Scan Workflow (Task 2) - -**Status:** WORKING ✓ - -- File: `panic-attacker/.github/workflows/scan-and-report.yml` -- Other repos can call: `uses: hyperpolymath/panic-attacker/.github/workflows/scan-and-report.yml@main` - -**Fixes applied:** -- SHA-pinned `dtolnay/rust-toolchain` action -- Removed non-existent `--format json` flag (panic-attack writes JSON automatically with `--output`) - -### 3. Pilot Repo Deployment (Task 3) - -**Status:** DEPLOYED ✓ - -All 3 pilot repos have `security-scan.yml` workflows: -- ✓ echidna -- ✓ ambientops -- ✓ verisimdb - -**Scan results:** -- echidna: 15 weak points (13 Medium, 2 High) -- ambientops: 0 weak points (CLEAN!) -- verisimdb: 12 weak points - -### 4. Hypatia VeriSimDB Connector (Task 4) - -**Status:** WORKING ✓ - -**Files created:** -- `lib/verisimdb_connector.ex` - Reads scans from verisimdb-data, transforms to Logtalk facts -- `lib/pattern_analyzer.ex` - Analyzes scans, generates summaries -- `prolog/pattern_detection.lgt` - Logtalk rules for pattern detection -- `mix.exs` - Project file with Jason dependency -- `test_integration.exs` - Integration test script - -**Verified working:** -- Loaded all 3 scans from verisimdb-data -- Generated summary: 27 total weak points across 3 repos -- Created Logtalk facts file: `/tmp/scan_facts.lgt` - -**Fixes applied:** -- Fixed field name mapping: panic-attack uses `"location"` not `"file"` - -### 5. Fleet Dispatcher (Task 5) - -**Status:** INITIAL ✓ - -**Files created:** -- `lib/fleet_dispatcher.ex` - Routes findings to sustainabot, echidnabot, rhodibot -- GraphQL mutations defined for each bot type - -**Current behavior:** -- Findings are logged (not yet sent to live bots) -- GraphQL endpoints need to be implemented by bot repos - ---- - -## 📊 Real Data Results - -### Scans Completed - -| Repo | Weak Points | Critical | High | Medium | Low | -|------|-------------|----------|------|--------|-----| -| echidna | 15 | 0 | 3 | 12 | 0 | -| ambientops | 0 | 0 | 0 | 0 | 0 | -| verisimdb | 12 | 0 | 0 | 12 | 0 | -| **Total** | **27** | **0** | **3** | **24** | **0** | - -### Pattern Detection - -Generated 27 Logtalk facts: -```prolog -weak_point('echidna', 'src/rust/ffi/mod.rs', 'UnsafeCode', 'High'). -weak_point('echidna', 'src/rust/provers/z3.rs', 'PanicPath', 'Medium'). -weak_point('verisimdb', 'rust-core/verisim-graph/src/lib.rs', 'PanicPath', 'Medium'). -... -``` - -### Notable Findings - -**echidna high-severity issues:** -- 7 unsafe blocks in `src/rust/ffi/mod.rs` (FFI boundary) -- 7 unsafe blocks in `src/rust/proof_search.rs` (optimization) -- 1 unsafe block in HOL tree-sitter bindings (expected) - -**ambientops:** Clean! Zero weak points. - -**verisimdb:** 12 unwrap/expect calls in modality stores (moderate risk) - ---- - -## 🛠️ Helper Tools Created - -### Manual Scan Ingestion - -**scripts/ingest-scan.sh** -```bash -./scripts/ingest-scan.sh echidna /tmp/echidna-scan.json -``` - -**scripts/scan-all.sh** -```bash -./scripts/scan-all.sh # scans echidna, ambientops, verisimdb -``` - -### Integration Testing - -**hypatia/test_integration.exs** -```bash -cd ~/Documents/hyperpolymath-repos/hypatia -mix run test_integration.exs -``` - ---- - -## 📚 Documentation Created - -### INTEGRATION.md (317 lines) - -Complete guide covering: -- Architecture diagram -- Current status and limitations -- Quick start (manual workflow) -- PAT setup instructions (for automated dispatch) -- Hypatia pattern detection examples -- Querying scan data -- Troubleshooting guide - -### scripts/README.md - -Documentation for helper scripts with examples. - ---- - -## ⚠️ Known Limitation - -### Automated Dispatch Blocked - -**Issue:** `GITHUB_TOKEN` cannot trigger `repository_dispatch` events in other repos (GitHub security policy). - -**Current workaround:** Manual scan ingestion using helper scripts. - -**Solution (for automated workflow):** - -1. Create Personal Access Token (PAT) with `repo` scope -2. Add as secret `VERISIMDB_PAT` in scanning repos -3. Update `scan-and-report.yml` to use PAT instead of GITHUB_TOKEN -4. See `INTEGRATION.md` for complete setup instructions - ---- - -## 📈 State Updates - -### verisimdb STATE.scm - -Updated with: -- GitHub CI integration: **COMPLETE (100%)** -- Hypatia pipeline: **INITIAL (40%)** -- Session history entry for 2026-02-08 - -### Repositories Updated - -**Commits & Pushes:** -- verisimdb-data: 5 commits (GitHub + GitLab) -- panic-attacker: 3 commits (GitHub) -- echidna: 1 commit (GitHub) -- ambientops: 1 commit (GitHub) -- verisimdb: 2 commits (GitHub) -- hypatia: 2 commits (GitHub) - -**Total:** 14 commits across 6 repos - ---- - -## 🎯 Next Steps (for Sonnet or future work) - -### Immediate -1. Configure PAT for automated dispatch (see INTEGRATION.md) -2. Add more repos to security scanning -3. Monitor weekly scans (scheduled via cron) - -### Short-term -4. Implement GraphQL endpoints in gitbot-fleet bots -5. Wire up live fleet dispatcher -6. Add temporal drift detection (compare scans over time) - -### Medium-term -7. SARIF output for GitHub Security tab integration -8. Automated rule learning (detect new patterns) -9. Real-time pattern alerts - -### Long-term -10. GitHub App for organization-wide scanning -11. Integration with echidnabot for formal verification -12. Automated fix generation via rhodibot - ---- - -## 📋 Quick Reference - -### Scan a repo manually -```bash -cd ~/Documents/hyperpolymath-repos/echidna -panic-attack assail . --output /tmp/echidna-scan.json -cd ~/Documents/hyperpolymath-repos/verisimdb-data -./scripts/ingest-scan.sh echidna /tmp/echidna-scan.json -git push && git push gitlab main -``` - -### Test Hypatia integration -```bash -cd ~/Documents/hyperpolymath-repos/hypatia -mix run test_integration.exs -``` - -### View scan results -```bash -cd ~/Documents/hyperpolymath-repos/verisimdb-data -jq '.repos' index.json -cat scans/echidna.json | less -``` - -### Trigger automated scan (requires PAT) -```bash -gh workflow run security-scan.yml --repo hyperpolymath/echidna -``` - ---- - -## ✨ Summary - -**Complete integration pipeline delivered:** -- ✅ Git-backed data storage (verisimdb-data) -- ✅ Reusable scan workflow (panic-attacker) -- ✅ 3 pilot repos scanning (echidna, ambientops, verisimdb) -- ✅ Pattern detection (hypatia) -- ✅ Fleet dispatcher (hypatia) -- ✅ Helper scripts for manual workflow -- ✅ Comprehensive documentation -- ✅ Verified with real data (27 weak points found) - -**Manual workflow fully operational.** Automated workflow requires PAT configuration (15 minutes of setup). - -**All code committed and pushed to GitHub + GitLab.** - ---- - ---- - -# Addendum: 2026-02-12 - Workflow Automation Update - -## Changes Made - -### Workflow File Updates (Steps 3-4 of PAT Setup) - -All workflow files have been updated to support automated cross-repo dispatch: - -1. **panic-attacker/scan-and-report.yml** (reusable workflow): - - Added `secrets:` block accepting optional `VERISIMDB_PAT` - - Dispatch token now uses `${{ secrets.VERISIMDB_PAT || secrets.GITHUB_TOKEN }}` fallback - - Added `-sf` flag to curl for silent failure detection - -2. **Caller workflows** (all 3 pilot repos updated): - - `echidna/.github/workflows/security-scan.yml` — passes VERISIMDB_PAT - - `ambientops/.github/workflows/security-scan.yml` — passes VERISIMDB_PAT - - `verisimdb/.github/workflows/security-scan.yml` — passes VERISIMDB_PAT - -### Git Mirroring Fixes - -- **ambientops**: Added GitLab remote, unshallowed clone, pushed successfully -- **echidna GitLab**: Diverged history (protected branch + expired PAT blocks force push) - -### Remaining Manual Step - -Only Steps 1-2 of the PAT setup remain: -1. Create classic PAT at https://github.com/settings/tokens with `repo` scope -2. Add as `VERISIMDB_PAT` secret to echidna, ambientops, and verisimdb repos - -Once done, automated scanning will work end-to-end. - ---- - -Generated: 2026-02-12T18:30:00Z diff --git a/TEST-NEEDS.adoc b/TEST-NEEDS.adoc new file mode 100644 index 00000000..40b2fda6 --- /dev/null +++ b/TEST-NEEDS.adoc @@ -0,0 +1,34 @@ +== TEST-NEEDS.md — verisimdb-data + +=== CRG Grade: C — ACHIEVED 2026-04-04 + +=== Current Test State + +[width="100%",cols="42%,29%,29%",options="header",] +|=== +|Category |Count |Notes +|Zig FFI tests |1 |`+ffi/zig/test/integration_test.zig+` +|Scorecard CI recipes |1 |`+recipes/recipe-scorecard-ci-tests.json+` +|Scan definitions |3 |Broad spectrum, proven repos, verified containers +|=== + +=== What’s Covered + +* [x] Zig FFI integration tests +* [x] Scorecard test recipes +* [x] Scan specifications for verification + +=== Still Missing (for CRG B+) + +* [ ] Data validation tests +* [ ] Database query tests +* [ ] Replication tests +* [ ] Performance benchmarks +* [ ] Data consistency tests + +=== Run Tests + +[source,bash] +---- +cd /var/mnt/eclipse/repos/verisimdb-data && cargo test +---- diff --git a/TEST-NEEDS.md b/TEST-NEEDS.md deleted file mode 100644 index e300f78c..00000000 --- a/TEST-NEEDS.md +++ /dev/null @@ -1,31 +0,0 @@ -# TEST-NEEDS.md — verisimdb-data - -## CRG Grade: C — ACHIEVED 2026-04-04 - -## Current Test State - -| Category | Count | Notes | -|----------|-------|-------| -| Zig FFI tests | 1 | `ffi/zig/test/integration_test.zig` | -| Scorecard CI recipes | 1 | `recipes/recipe-scorecard-ci-tests.json` | -| Scan definitions | 3 | Broad spectrum, proven repos, verified containers | - -## What's Covered - -- [x] Zig FFI integration tests -- [x] Scorecard test recipes -- [x] Scan specifications for verification - -## Still Missing (for CRG B+) - -- [ ] Data validation tests -- [ ] Database query tests -- [ ] Replication tests -- [ ] Performance benchmarks -- [ ] Data consistency tests - -## Run Tests - -```bash -cd /var/mnt/eclipse/repos/verisimdb-data && cargo test -``` diff --git a/TOPOLOGY.adoc b/TOPOLOGY.adoc new file mode 100644 index 00000000..7c45855a --- /dev/null +++ b/TOPOLOGY.adoc @@ -0,0 +1,52 @@ +== TOPOLOGY.md — verisimdb-data + +=== Purpose + +This repository serves two explicit purposes per +link:docs/decisions/ADR-0001-repo-purpose.adoc[`+docs/decisions/ADR-0001-repo-purpose.adoc+`]: + +[arabic] +. *Flat-file data store* for panic-attacker scan results, +hardware-crash-team findings, and drift snapshots received via GitHub +Actions `+workflow_dispatch+` events. Maintains a master index for +historical analysis of repository health and compliance drift over time. +. *ABI dogfood* for the hyperpolymath Idris2 + Zig ABI standard (shared +with `+proven+`, `+burble+`, `+gossamer+`). Lives in `+ffi/zig/+`. + +=== Module map + +.... +verisimdb-data/ +├── scans/ # panic-attacker scan results per repo (data) +├── dispatch/ # dispatch records (data) +├── patterns/ # drift / scan pattern definitions (data) +├── recipes/ # ingest and aggregation recipes (data) +├── outcomes/ # outcome records (data) +├── policy/ # storage and retention policy notes (data) +├── health/ # health-state snapshots (data) +├── index.json # master index of stored data (data) +├── ffi/ +│ └── zig/ # Zig FFI implementation (ABI dogfood) +├── scripts/ # ingest + index regen scripts (data) +├── docs/ +│ └── decisions/ # ADRs +└── .github/workflows/ # ingest + governance workflows +.... + +=== Data flow (Purpose 1) + +.... +[Workflow Dispatch] ──► [Ingest Handler] ──► [JSON Validation] ──► [File Storage] + │ + [Index Regen on push] ──► [Query Ready] +.... + +=== Integration points + +* *panic-attacker*: Upstream scanner sending results into `+scans/+`. +* *hardware-crash-team*: Hardware failure analysis (Purpose 1 consumer). +* *Drift detection*: Compliance change tracking against `+scans/+` +history. +* *verisimiser*: Consumes scan/drift data when wired to this repo as its +sidecar storage. Not a runtime dependency. +* *hyperpolymath CI/CD*: Automated data aggregation. diff --git a/TOPOLOGY.md b/TOPOLOGY.md deleted file mode 100644 index e06e88c2..00000000 --- a/TOPOLOGY.md +++ /dev/null @@ -1,54 +0,0 @@ - - - -# TOPOLOGY.md — verisimdb-data - -## Purpose - -This repository serves two explicit purposes per -[`docs/decisions/ADR-0001-repo-purpose.adoc`](docs/decisions/ADR-0001-repo-purpose.adoc): - -1. **Flat-file data store** for panic-attacker scan results, - hardware-crash-team findings, and drift snapshots received via - GitHub Actions `workflow_dispatch` events. Maintains a master - index for historical analysis of repository health and compliance - drift over time. -2. **ABI dogfood** for the hyperpolymath Idris2 + Zig ABI standard - (shared with `proven`, `burble`, `gossamer`). Lives in `ffi/zig/`. - -## Module map - -``` -verisimdb-data/ -├── scans/ # panic-attacker scan results per repo (data) -├── dispatch/ # dispatch records (data) -├── patterns/ # drift / scan pattern definitions (data) -├── recipes/ # ingest and aggregation recipes (data) -├── outcomes/ # outcome records (data) -├── policy/ # storage and retention policy notes (data) -├── health/ # health-state snapshots (data) -├── index.json # master index of stored data (data) -├── ffi/ -│ └── zig/ # Zig FFI implementation (ABI dogfood) -├── scripts/ # ingest + index regen scripts (data) -├── docs/ -│ └── decisions/ # ADRs -└── .github/workflows/ # ingest + governance workflows -``` - -## Data flow (Purpose 1) - -``` -[Workflow Dispatch] ──► [Ingest Handler] ──► [JSON Validation] ──► [File Storage] - │ - [Index Regen on push] ──► [Query Ready] -``` - -## Integration points - -- **panic-attacker**: Upstream scanner sending results into `scans/`. -- **hardware-crash-team**: Hardware failure analysis (Purpose 1 consumer). -- **Drift detection**: Compliance change tracking against `scans/` history. -- **verisimiser**: Consumes scan/drift data when wired to this repo as - its sidecar storage. Not a runtime dependency. -- **hyperpolymath CI/CD**: Automated data aggregation. diff --git a/docs/reports/audit/audit-2026-04-04.adoc b/docs/reports/audit/audit-2026-04-04.adoc new file mode 100644 index 00000000..c75fbbb8 --- /dev/null +++ b/docs/reports/audit/audit-2026-04-04.adoc @@ -0,0 +1,132 @@ +== Audit Report — verisimdb-data (2026-04-04) + +=== Summary + +VeriSimDB Data is a Git-backed flat-file storage repository for scan +results and drift detection data. The codebase functions as a data +lake/index for other projects’ security audit results. It exhibits +strong RSR compliance with comprehensive CI/CD automation. This is a +data repository, not a runtime codebase, so dangerous pattern concerns +are minimal and primarily confined to recipe documentation. + +=== Findings + +==== Critical + +None identified. This is a data aggregation repo with no runtime +execution code. + +==== High + +* *Dangerous pattern references in recipes/registry.json* — +Documentation-only references to believe_me, unsafePerformIO, +unsafeCoerce patterns from upstream projects +* These are SCAN RESULTS (not actual code) — patterns documented in +registry.json are findings from audited projects + +==== Medium + +* Recipe documentation is comprehensive but only examples/guidance (not +executed) +* Test coverage limited to ingest workflows +* Drift detection data is time-series snapshots + +=== RSR Compliance + +* *EXPLAINME.adoc*: present ✓ +* *0-AI-MANIFEST.a2ml*: present ✓ +* *.machine_readable/*: present ✓ +* *SECURITY.md*: present ✓ +* *CONTRIBUTING.md*: present ✓ + +=== Test Coverage + +VeriSimDB-data has minimal active test code (it’s a data repo), but +includes: + +* *Workflow validation*: ingest.yml (GitHub Actions triggered on +workflow_dispatch) +* *Data schema validation*: JSON validation in ingest workflow +* *Index consistency*: jekyll.yml, jekyll-gh-pages.yml for documentation +builds +* *CI workflows*: 17 quality, security, and policy enforcement workflows + +Test coverage is appropriate for a data repository: - recipes/ — +Refactoring recipes (documentation, not executable) - +patterns/registry.json — Scan result index (data, not code) - scans/ — +Historical scan snapshots (data artifacts) - drift/ — Drift detection +records (time-series data) + +=== Proof Debt + +*Status: N/A FOR DATA REPO* + +This is a *data aggregation repository*, not a proof-bearing codebase. +Dangerous patterns found in registry.json are: + +[arabic] +. *Scan result documentation* (from audited upstream repos): +* 14 believe_me instances in src/Proven/SafeEmail/Proofs.idr (remote +project) +* 18 believe_me instances in src/Proven/SafeJWT/Proofs.idr (remote +project) +* 6 believe_me instances in src/abi/Layout.idr (remote project) +* … (total 87 indexed patterns across 10+ projects) +. *Recipes* (documentation/guidance): +* recipe-remove-believe-me.json — How to eliminate believe_me +* recipe-unsafe-type-coercion.json — How to eliminate unsafe casts +* These are *guidance documents*, not actual code violations + +*Conclusion*: VeriSimDB-data correctly stores and documents dangerous +patterns found in OTHER projects’ codebase scans. This is the intended +use case. + +=== Publication Safety + +*Claim*: "`Data aggregation repository for security audit results`" + +*Evidence*: - registry.json documents patterns from scanned repositories +- scans/ contains JSON results from panic-attack and other tools - +Recipes provide remediation guidance (best practices) - Index is +machine-readable for downstream tool consumption + +This is appropriate. The repo transparently documents findings from +external audits. + +=== CI Health + +*Status: COMPREHENSIVE* + +Active workflows (17 files): - ingest.yml — Data ingestion trigger - +mirror.yml — Git repository mirroring - codeql.yml, security-policy.yml +— Security scanning - quality.yml — Code quality - rsr-antipattern.yml — +RSR policy enforcement - npm-bun-blocker.yml, ts-blocker.yml, +guix-nix-policy.yml — Language/tool enforcement - scorecard.yml, +secret-scanner.yml — Supply chain security - jekyll.yml, +jekyll-gh-pages.yml — Documentation - workflow-linter.yml, +wellknown-enforcement.yml — Standards compliance - instant-sync.yml — +Mirroring + +*No failures detected*. All workflows properly configured. + +=== Verdict + +*PUBLISHABLE NOW* + +VeriSimDB-data is functioning as intended: a transparent data lake +documenting security findings across the estate. The dangerous patterns +found in registry.json are *correctly* documented as findings from OTHER +projects’ scans—not code defects in this repo itself. + +Publication is appropriate because: 1. Data aggregation is the intended +use case 2. Scan results are transparently documented 3. Remediation +recipes are provided 4. Complete RSR compliance 5. Comprehensive CI +enforcement + +The fact that dangerous patterns appear in scans is NOT a defect—it’s +evidence that the audit pipeline is working correctly. + +''''' + +*Audited by*: M2 estate audit (2026-04-04) *Confidence*: HIGH +*Recommendation*: SHIP AS-IS diff --git a/docs/reports/audit/audit-2026-04-04.md b/docs/reports/audit/audit-2026-04-04.md deleted file mode 100644 index 53667a33..00000000 --- a/docs/reports/audit/audit-2026-04-04.md +++ /dev/null @@ -1,115 +0,0 @@ -# Audit Report — verisimdb-data (2026-04-04) - -## Summary - -VeriSimDB Data is a Git-backed flat-file storage repository for scan results and drift detection data. The codebase functions as a data lake/index for other projects' security audit results. It exhibits strong RSR compliance with comprehensive CI/CD automation. This is a data repository, not a runtime codebase, so dangerous pattern concerns are minimal and primarily confined to recipe documentation. - -## Findings - -### Critical - -None identified. This is a data aggregation repo with no runtime execution code. - -### High - -- **Dangerous pattern references in recipes/registry.json** — Documentation-only references to believe_me, unsafePerformIO, unsafeCoerce patterns from upstream projects -- These are SCAN RESULTS (not actual code) — patterns documented in registry.json are findings from audited projects - -### Medium - -- Recipe documentation is comprehensive but only examples/guidance (not executed) -- Test coverage limited to ingest workflows -- Drift detection data is time-series snapshots - -## RSR Compliance - -- **EXPLAINME.adoc**: present ✓ -- **0-AI-MANIFEST.a2ml**: present ✓ -- **.machine_readable/**: present ✓ -- **SECURITY.md**: present ✓ -- **CONTRIBUTING.md**: present ✓ - -## Test Coverage - -VeriSimDB-data has minimal active test code (it's a data repo), but includes: - -- **Workflow validation**: ingest.yml (GitHub Actions triggered on workflow_dispatch) -- **Data schema validation**: JSON validation in ingest workflow -- **Index consistency**: jekyll.yml, jekyll-gh-pages.yml for documentation builds -- **CI workflows**: 17 quality, security, and policy enforcement workflows - -Test coverage is appropriate for a data repository: -- recipes/ — Refactoring recipes (documentation, not executable) -- patterns/registry.json — Scan result index (data, not code) -- scans/ — Historical scan snapshots (data artifacts) -- drift/ — Drift detection records (time-series data) - -## Proof Debt - -**Status: N/A FOR DATA REPO** - -This is a **data aggregation repository**, not a proof-bearing codebase. Dangerous patterns found in registry.json are: - -1. **Scan result documentation** (from audited upstream repos): - - 14 believe_me instances in src/Proven/SafeEmail/Proofs.idr (remote project) - - 18 believe_me instances in src/Proven/SafeJWT/Proofs.idr (remote project) - - 6 believe_me instances in src/abi/Layout.idr (remote project) - - ... (total 87 indexed patterns across 10+ projects) - -2. **Recipes** (documentation/guidance): - - recipe-remove-believe-me.json — How to eliminate believe_me - - recipe-unsafe-type-coercion.json — How to eliminate unsafe casts - - These are **guidance documents**, not actual code violations - -**Conclusion**: VeriSimDB-data correctly stores and documents dangerous patterns found in OTHER projects' codebase scans. This is the intended use case. - -## Publication Safety - -**Claim**: "Data aggregation repository for security audit results" - -**Evidence**: -- registry.json documents patterns from scanned repositories -- scans/ contains JSON results from panic-attack and other tools -- Recipes provide remediation guidance (best practices) -- Index is machine-readable for downstream tool consumption - -This is appropriate. The repo transparently documents findings from external audits. - -## CI Health - -**Status: COMPREHENSIVE** - -Active workflows (17 files): -- ingest.yml — Data ingestion trigger -- mirror.yml — Git repository mirroring -- codeql.yml, security-policy.yml — Security scanning -- quality.yml — Code quality -- rsr-antipattern.yml — RSR policy enforcement -- npm-bun-blocker.yml, ts-blocker.yml, guix-nix-policy.yml — Language/tool enforcement -- scorecard.yml, secret-scanner.yml — Supply chain security -- jekyll.yml, jekyll-gh-pages.yml — Documentation -- workflow-linter.yml, wellknown-enforcement.yml — Standards compliance -- instant-sync.yml — Mirroring - -**No failures detected**. All workflows properly configured. - -## Verdict - -**PUBLISHABLE NOW** - -VeriSimDB-data is functioning as intended: a transparent data lake documenting security findings across the estate. The dangerous patterns found in registry.json are **correctly** documented as findings from OTHER projects' scans—not code defects in this repo itself. - -Publication is appropriate because: -1. Data aggregation is the intended use case -2. Scan results are transparently documented -3. Remediation recipes are provided -4. Complete RSR compliance -5. Comprehensive CI enforcement - -The fact that dangerous patterns appear in scans is NOT a defect—it's evidence that the audit pipeline is working correctly. - ---- - -**Audited by**: M2 estate audit (2026-04-04) -**Confidence**: HIGH -**Recommendation**: SHIP AS-IS diff --git a/docs/tech-debt-2026-05-26.adoc b/docs/tech-debt-2026-05-26.adoc new file mode 100644 index 00000000..9e514b68 --- /dev/null +++ b/docs/tech-debt-2026-05-26.adoc @@ -0,0 +1,71 @@ +== Tech-Debt Audit — verisimdb-data — 2026-05-26 + +*Source:* estate-wide automated scan 2026-05-26. *Companion:* +https://github.com/hyperpolymath/standards/tree/main/docs/audits[`+hyperpolymath/standards+` +2026-05-26-estate-*-debt audits]. *Combined severity:* `+MEDIUM+`. + +This file records the _raw findings_ — it does not by itself fix the +debt. Each section ends with a '`Recommended next move`' line; closing +the debt is follow-up work. + +=== 1. Proof debt + +No proof-bearing files (`+*.v+`, `+*.lean+`, `+*.agda+`, `+*.idr+`, +`+*.idr2+`, `+*.fst+`, `+*.dfy+`, `+*.tla+`, `+*.ads+`, `+*.adb+`) found +in this repo. + +*Recommended next move:* none. + +=== 2. Licence debt + +[cols=",",options="header",] +|=== +|Field |Value +|LICENSE file |`+LICENSE+` +|SPDX header |`+MPL-2.0+` +|Manifest licence |`+NONE+` +|Body classifier |`+Palimp-MPL-2.0+` +|Severity |`+ok+` +|=== + +*Recommended next move:* none for licence. + +=== 3. Documentation debt + +[cols=",",options="header",] +|=== +|Field |Value +|README lines |117 +|`+docs/+` files |5 +|`+docs/+` LoC |483 +|CHANGELOG.md |N +|CONTRIBUTING.md |Y +|CODE_OF_CONDUCT.md |Y +|SECURITY.md |Y +|Severity |`+MEDIUM+` +|=== + +*Recommended next move:* introduce a `+docs/+` directory. The README at +117 lines has likely grown to do the work of `+docs/+` — split it into a +thin README + `+docs/architecture.md+`, `+docs/usage.md+`, etc. +Heavy-wiki exemplars to copy from: `+affinescript+`, `+boj-server+`, +`+echidna+`, `+hypatia+`. + +Additionally: *CHANGELOG.md is missing.* 65% of estate repos lack one — +adopting a CHANGELOG (or auto-generating via `+git-cliff+`) is a +recommended estate-wide follow-up. + +=== Cross-references + +* Estate proof-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-proof-debt.md+` +* Estate licence-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-licence-debt.md+` +* Estate documentation-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-documentation-debt.md+` + +''''' + +🤖 Generated by Claude Code estate-wide tech-debt scan (2026-05-26). +This file is informational — closing the debt is follow-up work owned by +the maintainer. diff --git a/docs/tech-debt-2026-05-26.md b/docs/tech-debt-2026-05-26.md deleted file mode 100644 index d8c1d2c8..00000000 --- a/docs/tech-debt-2026-05-26.md +++ /dev/null @@ -1,57 +0,0 @@ - - -# Tech-Debt Audit — verisimdb-data — 2026-05-26 - -**Source:** estate-wide automated scan 2026-05-26. -**Companion:** [`hyperpolymath/standards` 2026-05-26-estate-*-debt audits](https://github.com/hyperpolymath/standards/tree/main/docs/audits). -**Combined severity:** `MEDIUM`. - -This file records the *raw findings* — it does not by itself fix the debt. Each section ends with a 'Recommended next move' line; closing the debt is follow-up work. - -## 1. Proof debt - -No proof-bearing files (`*.v`, `*.lean`, `*.agda`, `*.idr`, `*.idr2`, `*.fst`, `*.dfy`, `*.tla`, `*.ads`, `*.adb`) found in this repo. - -**Recommended next move:** none. - -## 2. Licence debt - -| Field | Value | -|---|---| -| LICENSE file | `LICENSE` | -| SPDX header | `MPL-2.0` | -| Manifest licence | `NONE` | -| Body classifier | `Palimp-MPL-2.0` | -| Severity | `ok` | - -**Recommended next move:** none for licence. - -## 3. Documentation debt - -| Field | Value | -|---|---| -| README lines | 117 | -| `docs/` files | 5 | -| `docs/` LoC | 483 | -| CHANGELOG.md | N | -| CONTRIBUTING.md | Y | -| CODE_OF_CONDUCT.md | Y | -| SECURITY.md | Y | -| Severity | `MEDIUM` | - -**Recommended next move:** introduce a `docs/` directory. The README at 117 lines has likely grown to do the work of `docs/` — split it into a thin README + `docs/architecture.md`, `docs/usage.md`, etc. Heavy-wiki exemplars to copy from: `affinescript`, `boj-server`, `echidna`, `hypatia`. - -Additionally: **CHANGELOG.md is missing.** 65% of estate repos lack one — adopting a CHANGELOG (or auto-generating via `git-cliff`) is a recommended estate-wide follow-up. - -## Cross-references - -- Estate proof-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-proof-debt.md` -- Estate licence-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-licence-debt.md` -- Estate documentation-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-documentation-debt.md` - ---- - -🤖 Generated by Claude Code estate-wide tech-debt scan (2026-05-26). This file is informational — closing the debt is follow-up work owned by the maintainer. diff --git a/llm-warmup-dev.adoc b/llm-warmup-dev.adoc new file mode 100644 index 00000000..a6a4d8e9 --- /dev/null +++ b/llm-warmup-dev.adoc @@ -0,0 +1,19 @@ +== LLM Warmup — verisimdb-data (Developer) + +=== What is verisimdb-data? + +See README.adoc for overview. + +=== Key Commands + +* `+just setup+` — set up development environment +* `+just build+` — build the project +* `+just test+` — run tests +* `+just doctor+` — diagnose issues +* `+just heal+` — attempt auto-repair + +=== Quick Context + +* License: MPL-2.0 +* Part of hyperpolymath ecosystem +* See EXPLAINME.adoc for architecture diff --git a/llm-warmup-dev.md b/llm-warmup-dev.md deleted file mode 100644 index 78c672f8..00000000 --- a/llm-warmup-dev.md +++ /dev/null @@ -1,16 +0,0 @@ -# LLM Warmup — verisimdb-data (Developer) - -## What is verisimdb-data? -See README.adoc for overview. - -## Key Commands -- `just setup` — set up development environment -- `just build` — build the project -- `just test` — run tests -- `just doctor` — diagnose issues -- `just heal` — attempt auto-repair - -## Quick Context -- License: MPL-2.0 -- Part of hyperpolymath ecosystem -- See EXPLAINME.adoc for architecture diff --git a/llm-warmup-user.adoc b/llm-warmup-user.adoc new file mode 100644 index 00000000..9a0b26e3 --- /dev/null +++ b/llm-warmup-user.adoc @@ -0,0 +1,19 @@ +== LLM Warmup — verisimdb-data (User) + +=== What is verisimdb-data? + +See README.adoc for overview. + +=== Key Commands + +* `+just setup+` — set up development environment +* `+just build+` — build the project +* `+just test+` — run tests +* `+just doctor+` — diagnose issues +* `+just heal+` — attempt auto-repair + +=== Quick Context + +* License: MPL-2.0 +* Part of hyperpolymath ecosystem +* See EXPLAINME.adoc for architecture diff --git a/llm-warmup-user.md b/llm-warmup-user.md deleted file mode 100644 index c4ffae66..00000000 --- a/llm-warmup-user.md +++ /dev/null @@ -1,16 +0,0 @@ -# LLM Warmup — verisimdb-data (User) - -## What is verisimdb-data? -See README.adoc for overview. - -## Key Commands -- `just setup` — set up development environment -- `just build` — build the project -- `just test` — run tests -- `just doctor` — diagnose issues -- `just heal` — attempt auto-repair - -## Quick Context -- License: MPL-2.0 -- Part of hyperpolymath ecosystem -- See EXPLAINME.adoc for architecture diff --git a/scripts/README.md b/scripts/README.adoc similarity index 54% rename from scripts/README.md rename to scripts/README.adoc index 12443e64..a6befb5b 100644 --- a/scripts/README.md +++ b/scripts/README.adoc @@ -1,17 +1,19 @@ -# SPDX-License-Identifier: CC-BY-SA-4.0 +== SPDX-License-Identifier: CC-BY-SA-4.0 -# VeriSimDB Data Scripts +== VeriSimDB Data Scripts Helper scripts for ingesting scan results into verisimdb-data. -## Scripts +=== Scripts -### `ingest-scan.sh` +==== `+ingest-scan.sh+` Ingest a single scan result into verisimdb-data. -**Usage:** -```bash +*Usage:* + +[source,bash] +---- # Run panic-attack on a repo cd ~/Documents/hyperpolymath-repos/echidna panic-attack assail . --output /tmp/echidna-scan.json @@ -19,25 +21,27 @@ panic-attack assail . --output /tmp/echidna-scan.json # Ingest the result cd ~/Documents/hyperpolymath-repos/verisimdb-data ./scripts/ingest-scan.sh echidna /tmp/echidna-scan.json -``` +---- + +*What it does:* 1. Copies scan result to `+scans/.json+` 2. +Updates `+index.json+` with summary 3. Commits the changes -**What it does:** -1. Copies scan result to `scans/.json` -2. Updates `index.json` with summary -3. Commits the changes +*After running:* -**After running:** -```bash +[source,bash] +---- git push git push gitlab main -``` +---- -### `scan-all.sh` +==== `+scan-all.sh+` Scan multiple repos and ingest all results in one go. -**Usage:** -```bash +*Usage:* + +[source,bash] +---- # Scan default repos (echidna, ambientops, verisimdb) cd ~/Documents/hyperpolymath-repos/verisimdb-data ./scripts/scan-all.sh @@ -47,64 +51,70 @@ cd ~/Documents/hyperpolymath-repos/verisimdb-data # Scan all repos in a directory ./scripts/scan-all.sh ~/Documents/hyperpolymath-repos/* -``` +---- + +*What it does:* 1. Runs `+panic-attack assail+` on each repo 2. Ingests +each result using `+ingest-scan.sh+` 3. Shows summary of all scans -**What it does:** -1. Runs `panic-attack assail` on each repo -2. Ingests each result using `ingest-scan.sh` -3. Shows summary of all scans +*After running:* -**After running:** -```bash +[source,bash] +---- git push git push gitlab main -``` +---- -## Integration with GitHub Actions +=== Integration with GitHub Actions -Once a Personal Access Token (PAT) is configured, the reusable workflow will automatically send scan results to verisimdb-data: +Once a Personal Access Token (PAT) is configured, the reusable workflow +will automatically send scan results to verisimdb-data: -```yaml +[source,yaml] +---- # In any repo's .github/workflows/security-scan.yml jobs: scan: uses: hyperpolymath/panic-attacker/.github/workflows/scan-and-report.yml@main secrets: PAT_TOKEN: ${{ secrets.VERISIMDB_PAT }} -``` +---- -**PAT Requirements:** -- Scope: `repo` (to trigger repository_dispatch events) -- Stored as secret `VERISIMDB_PAT` in each scanning repo -- Alternative: Use GitHub App with repository_dispatch permissions +*PAT Requirements:* - Scope: `+repo+` (to trigger repository_dispatch +events) - Stored as secret `+VERISIMDB_PAT+` in each scanning repo - +Alternative: Use GitHub App with repository_dispatch permissions -## Querying Results +=== Querying Results -### View index -```bash +==== View index + +[source,bash] +---- jq '.' index.json -``` +---- + +==== Find repos with high weak point counts -### Find repos with high weak point counts -```bash +[source,bash] +---- jq -r '.repos | to_entries | map(select(.value.weak_points > 10)) | map("\(.key): \(.value.weak_points)") | .[]' index.json -``` +---- + +==== Get latest scan time -### Get latest scan time -```bash +[source,bash] +---- jq -r '.last_updated' index.json -``` +---- -## Hypatia Integration +=== Hypatia Integration Test pattern detection with current scan data: -```bash +[source,bash] +---- cd ~/Documents/hyperpolymath-repos/hypatia mix run test_integration.exs -``` +---- -This will: -- Load all scans from verisimdb-data -- Generate Logtalk facts +This will: - Load all scans from verisimdb-data - Generate Logtalk facts - Show summary statistics