Skip to content

cybedtools 0.4.0 - #10

Merged
ryanstraight merged 72 commits into
mainfrom
dev-0.4.0
Sep 21, 2026
Merged

ryanstraight merged 72 commits into
mainfrom
dev-0.4.0

Conversation

@ryanstraight

Copy link
Copy Markdown
Owner

Release 0.4.0.

  • Four frameworks: CSTA PK-12 CS (2026), DigComp 3.0 (replacing 2.2), SCyWF 1.5:2026 (NCA, verbatim), CyBOK v1.1.0.
  • CCSSF now published in full with the Canadian Centre for Cyber Security's permission.
  • New API: framework_slug on all query results, unit_element_bindings() (role_element_bindings() deprecated), unit_relation_bindings(), framework_similarity(), cybed_fetch() and load_graph() over per-framework data releases.
  • Python package 0.1.0 under python/, tested against shared R-generated conformance goldens.
  • Documentation figures computed at render time. Concordance: new framework pages, DigComp 3.0, llms.txt.
  • Title: A Package for Reproducible Analysis of Cybersecurity Workforce and Learning Frameworks.
  • Data release 2026.09.2 manifest config (assets published separately after merge).

Matches OTCCF's treatment: standards are published statements, and a
parser-split fragment would be an invented unit rather than one CSTA
printed. Parent elements unaffected; the 45 parsed subpoints drop out.
Source of record is the official JSON-LD data supplement, hash-anchored
and verified at ingest, replacing the PDF scrape. Versionless IRIs
survive (digcomp:AREA-*, digcomp:COMP-x.y); elements move down to the
362 Competence Statements, so competences become a second organizing-
unit tier alongside the 5 areas. Learning Outcomes (522 post-errata)
attach as Examples on their Competence. 2.2 raw stays archived under
data/raw/digcomp/v2.2/.
E5-E7 are published as instructions without replacement text; composing
new sentences from them put our words in JRC's mouth. E1-E4 keep JRC's
own quoted text. Learning outcome count unaffected, still 522 from E1.
Adds a generic errata section to the ingestion summary generator.
Matches OTCCF and csta-2026: Competence Statement text yields fragments
like clickbait, validating, AI systems rather than published enumerations.
Parents unchanged at 362; examples unchanged at 522. Updates invariants,
bands, and roxygen counts accordingly.
DESCRIPTION, CITATION.cff, and context7.json still named DigComp 2.2.
The cross-framework vignette hand-typed a 21-elements-across-5-areas
density figure that was never true for 3.0. Rewrites that paragraph to
drop hand-typed per-framework numbers and point to the computed table
and framework_summary above it instead.
Acknowledges the digcomp-3.0 subpoint drift left by 5a9bafe, which disabled its parser without a rebuild.
PDF encoding drops the space in both the tree text layer and the
independent verification extraction for AC-02.4 and AC-02.5. Add a
documented, closed verbatim-check exception list (owner decision) so
the printed form is carried without loosening the check.
Move the four framework additions/upgrades under a (development
version) heading above the released 0.3.1 section, which stays as
published. dev-0.4.0 branch, rebased cybok-ingest stack onto
origin/main (0.3.1).
Owner decision 2026-09-21: the Canadian Centre for Cyber Security gave
written permission for full-text publication with attribution,
superseding the 2026-09-19 reference-only reading of its reply.

- docs/framework-invariants.yml: ccssf policy structure_only -> full_with_attribution
- framework_licenses: ccssf-2022 granted = TRUE, license_short updated
- creditText/license wording updated everywhere it read "Referenced as
  the Canadian Centre for Cyber Security asked"
- tests updated to the new truth; guard in build-framework-licenses.R
  now checks permission IS claimed, not that it isn't
- data-release.yml (already-cut release 2026.09.1) left unchanged
# Conflicts:
#	NEWS.md
Base URL now points at releases/download/data-v<version>, matching how
scripts/030-export-release.R actually names files (<slug>.nt.gz +
manifest.json). A {version} placeholder in the base URL is substituted
in place (a release tag has no separate version segment); every other
base URL (a mirror, or the mock release used in tests) keeps the old
<base>/<version>/<file> layout via cybed_release_url().

Regenerated inst/conformance/ goldens: cybed_license.csv and
framework_summary.csv change (they track real package data, now
including cybok/scywf/csta-2026/digcomp-3.0/ccssf's permission
update); every fixture-derived golden is byte-identical.
Windows checkouts with core.autocrlf=true were silently converting the
conformance goldens' LF line endings to CRLF, tripping the CR-byte
guard in test-conformance-goldens.R even though nothing about the
data changed -- a repo gap, not a data regression.
Foundation port: cybed_fetch/load_graph mirroring R release resolution
and sha256 verification, framework_summary/framework_licenses/cybed_license
from shipped CSVs, rdflib default graph backend with pyoxigraph extra,
and a conformance test harness with skipped placeholders for the
not yet implemented query helpers. CI matrix now covers 3.10 to 3.13.
…milarity graph walks

R: extend 040-build-goldens.R with 6 new goldens, document format and regen/sync steps in README. Python: assert new goldens in test_queries.py, cover them in the drift check, fix role/organizing_unit_framework_bindings column order to match the golden contract, and rewire similarity.py to use the public query functions instead of private triple-walking copies.
Framework pages hand-typed organizing-unit and element counts in
frontmatter, Counts tables, and prose. Removed the frontmatter keys
(they only fed the frameworks listing, now a reactable table built
from framework_summary in R) and replaced every hand-typed Counts
table and count sentence with inline R against framework_summary.

For counts framework_summary does not carry (DigComp's area and
competence split, SCyWF's category and specialty-area split, CCSSF's
core vs adjacent role split, OTCCF's TSC and career-track counts,
SFIA's level count), added a data-prep script that queries the
combined RDF graph once and caches the results.

Fixed several numbers that were wrong: DCWF's slug was misdocumented
as dcwf-v51 instead of dcwf-v5.1, a K-12 query cited 20 CSTA
subpoints where the graph has 24, and a query cross-link called
k12-alignment.qmd a four-framework comparison when it compares two.

install.qmd's pasted static tibble output is now a live chunk.

A few numbers (DigComp's pre-errata learning-outcome count, its
per-name revision counts, OTCCF's proficiency-grid level count) have
no render-time source and were rewritten without the figure instead
of computed.
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 93.85475% with 11 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
R/cybed-fetch.R 88.76% 10 Missing ⚠️
R/similarity-helpers.R 97.87% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@ryanstraight
ryanstraight merged commit ca891ac into main Sep 21, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants