feat(compliance): add RMF/FedRAMP/DoD ATO package automation skill - #241
ForeverYoung48 wants to merge 6 commits into
Conversation
Introduces `.gemini/skills/compliance/`, a self-contained agent skill that
generates Authorization to Operate (ATO) packages from the Terraform in this
repository. It extracts live architecture facts from a blueprint, hydrates
authoritative templates, and emits the SSP, 20 policy manuals, SCTM, PPSM,
HW/SW inventory, POA&M, FIPS 140-3 matrix, IR runbooks, NIST OSCAL packages
and the master Path to Authorization roadmap as Markdown, Word, YAML and
macro-enabled Excel.
Targets NIST SP 800-53 Rev. 5, FedRAMP Moderate/High, DoD CC SRG IL4/IL5/IL6,
StateRAMP, CJIS and FISMA.
This is the first agent skill in the repository, so it also establishes the
`.gemini/skills/<skill-name>/` convention: each skill is defined by a SKILL.md
with YAML frontmatter and resolves its own root at runtime rather than
hardcoding paths or depending on a checkout location.
No Terraform, blueprint or GCP resource is changed by this commit. The skill
is local, read-only tooling.
Adaptations for this repository
-------------------------------
* Retarget the system-name inference denylist from the source repository's
layout to ours (workspace, stellar-engine, blueprints, modules, fast), so
blueprints no longer infer a generic "Blueprints Platform" system name.
* Register the skill in GEMINI.md: setup, the three-command operational
workflow, target-folder isolation rules, and the python-hcl2 pin rationale.
* Ignore `**/.venv/` so the skill's isolated virtualenv stays out of the tree.
Correctness fixes to the extraction engine
------------------------------------------
* Route parsed Terraform through the structured extraction path. The
classifier branches on `isinstance(body, dict)`, but the parsed AST was
wrapped in HclBlock, a `str` subclass, so the gate never matched and every
.tf file silently fell back to substring heuristics. That discarded CMEK key
identity, reporting only that some key existed rather than which one, and
dropped module-declared resources from the accreditation boundary, because
the fallback early-returns on any name containing `${`.
* Resolve `var.*` references across the parsed AST. The structured path reads
attributes directly and, unlike the text path's extract_hcl_attr, performed
no interpolation, surfacing values such as `${var.vm_size}` as machine types.
* Resolve asset identifiers through one shared helper across all ten asset
categories, so no name reaches a deliverable as raw expression syntax. Where
a reference cannot be resolved statically, prefer the Terraform logical name
over a stripped fragment such as "bucket", which is generic, collision-prone
and indistinguishable from a real name.
* Emit compute instance exposure as `has_public_ip`. The structured path used
`has_pub_ip`, so the POA&M rule for internet-exposed instances, which tests
`has_public_ip is True`, could never fire for anything ingested structurally,
including via the recommended `terraform show -json` path.
* Tolerate scalar `protocol` and `ports` firewall values. A single-element
variable default such as `[5432]` is stored unwrapped, so `ports = var.x`
arrives as an int; iterating it raised TypeError and aborted extraction for
an entire blueprint rather than degrading on one field.
* State an unresolvable CMEK reference in words instead of stripping
`${var.kms_key_name}` down to `kms_key_name`, which reads as a real key name
in the SSP.
* Make two HCL parser tests backend-aware. They encoded the in-repo fallback
parser's behaviour and passed only when python-hcl2 was absent.
Verification
------------
Measured across all 43 leaf blueprints: 43/43 extract with no failures and no
HCL parse gaps; asset names containing unresolved expressions drop from 2 to 0
while the inventory retains all 52 asset items. Engine suite is 386/386, with
10 new regression tests covering CMEK key survival, boundary completeness for
expression-named resources, expression-free identifiers, variable resolution,
the POA&M field name and firewall shape tolerance. A full
extract/generate/validate run against blueprints/il5/bigquery produces 71
schema-valid deliverables with 0 unresolved tokens.
Known limitation
----------------
Module-created infrastructure is not visible to static HCL scanning, so
blueprints whose resources live behind `module` blocks report few or no assets.
Driving the skill from `terraform show -json` output captures them, arriving
through the same structured path fixed here.
Implements the outcome of a full review of the ATO automation skill. The changes fall into five groups. Truth in extraction (the central issue) The extractor defaulted unstated security attributes to their compliant value, so a blueprint that never mentioned a control was documented in the generated SSP as enforcing it. Several of those defaults also contradicted the fabric modules being invoked: modules/gcs defaults versioning to null and modules/cloudsql-instance defaults backup_configuration.enabled to false, yet both were reported as enabled. KMS keys with no version_template were reported as HSM, which fabricates a FIPS 140-3 Level 3 claim at IL5. Attributes now resolve to the real provider or module default, or to an explicit None when the degraded text-scan path cannot determine them. A new DATA_GAP POA&M rule turns those unknowns into tracked findings, and deliberately ignores explicit False values so a deficiency is never double-reported under two identifiers. Against blueprints/il5/postgresql this correctly surfaces an unenforced TLS posture that was previously reported as compliant. Safety of the automation terraform plan is gated behind --allow-terraform-plan and runs with -lock=false -refresh=false; syft and trivy behind --allow-scanners. Semgrep defaults to the bundled offline ruleset, always passes --metrics=off, and refuses cleartext HTTP rulesets. Dependency bootstrapping is fail-closed via COMPLIANCE_ALLOW_BORROWED_DEPS. Mock-detection branches were removed from production code in favour of dependency injection. Binary writers gained boundary enforcement, symlink refusal, zip-bomb guards, and atomic replace. Correctness of generated artifacts allowed_boundary is now passed into export_document rather than only asserted afterwards; artifact enumeration is symlink-safe; --fix takes a timestamped backup before regenerating. Provenance and privacy Stripped third-party Microsoft 365 tenant GUIDs from two eMASS templates and documented origin, hashes, and modifications in templates/PROVENANCE.md. Sanitized vendor names from shipped defaults. Repository hygiene Removed duplicate compliance_skill.md, corrected unverifiable test-count and efficacy claims, added Apache headers, and added a CI job so the engine suite actually runs (pytest.ini's testpaths never reached it). Suite: 407 tests passing on Python 3.9, the declared floor. Verified end to end on blueprints/il5/bigquery: 71 deliverables, 0 unresolved tokens, all schemas valid.
Removes all 489 emoji occurrences from the 38 files this PR adds, covering
console output, markdown documentation, and the policy, SSP, POA&M, PPSM,
HW/SW and FIPS templates that are hydrated into submitted ATO deliverables.
Three places had the emoji load-bearing rather than decorative:
- template_engine.py emitted badges as "<mark ...>WARNING [TYPE: label]</mark>".
The bracketed marker already identifies the badge, so only the glyph was
dropped. Tests asserting on those literals track the same change.
- Three tests asserted assertNotIn on a bare emoji. Stripping the literal in
place would have left assertNotIn("") which passes against any input, so
they were removed; each was paired with an assertNotIn("<mark") that already
asserts the real property.
- validate_compliance_artifacts.py had a .replace() of the badge glyph, now a
no-op and removed. docx_generator.py's callout classifier keys off text
keywords, not glyphs, so only its titles and one regex prefix changed.
Deliberately left alone: the checkmark in Terraform variable tables under
fast/stages-aw/. That marker is emitted by tools/tfdoc.py, appears in 136
READMEs on main, and is regenerated by tooling, so changing it here would be
both inconsistent and transient.
Suite: 407 tests passing on Python 3.9. Verified by regenerating the full
package for blueprints/il5/bigquery and scanning all 73 deliverables,
including the OOXML inside .docx and .xlsm: zero emoji, and audit results
identical to before the change.
|
Ran an in-depth review and hardening pass across the compliance skill to get the PR ready for production and open source use. Here is a summary of the improvements and cleanups applied across the branch: Accurate extractionPreviously, the extractor defaulted missing or unstated Terraform settings to compliant values (like assuming shielded VMs, SSL, backups, or HSM protection were on). Updated these to reflect genuine provider defaults, or explicit null values when static analysis cannot determine them. Also introduced a new DATA_GAP finding in Operational safetyExternal tooling executions are now strictly controlled. Terraform plan no longer runs automatically on initialized workspaces; it is gated behind the Security and file protectionsAll binary document exporters and hydrators now enforce path boundary checks directly within Open source readinessReplaced hardcoded author strings with a configurable Code cleanup and optimizationFixed missing imports (such as VerificationAll 407 tests pass on Python 3.9 in under 50 seconds (with 2 skipped and 0 failures). An end-to-end run against |
aghassemlouei
added
enhancement
3 participants
Introduces
.gemini/skills/compliance/, a self-contained agent skill that generates Authorization to Operate (ATO) packages from the Terraform in this repository. It extracts live architecture facts from a blueprint, hydrates authoritative templates, and emits the SSP, 20 policy manuals, SCTM, PPSM, HW/SW inventory, POA&M, FIPS 140-3 matrix, IR runbooks, NIST OSCAL packages and the master Path to Authorization roadmap as Markdown, Word, YAML and macro-enabled Excel.Targets NIST SP 800-53 Rev. 5, FedRAMP Moderate/High, DoD CC SRG IL4/IL5/IL6, StateRAMP, CJIS and FISMA.
This is the first agent skill in the repository, so it also establishes the
.gemini/skills/<skill-name>/convention: each skill is defined by a SKILL.md with YAML frontmatter and resolves its own root at runtime rather than hardcoding paths or depending on a checkout location.No Terraform, blueprint or GCP resource is changed by this commit. The skill is local, read-only tooling.
Adaptations for this repository
**/.venv/so the skill's isolated virtualenv stays out of the tree.Correctness fixes to the extraction engine
Route parsed Terraform through the structured extraction path. The classifier branches on
isinstance(body, dict), but the parsed AST was wrapped in HclBlock, astrsubclass, so the gate never matched and every .tf file silently fell back to substring heuristics. That discarded CMEK key identity, reporting only that some key existed rather than which one, and dropped module-declared resources from the accreditation boundary, because the fallback early-returns on any name containing${.Resolve
var.*references across the parsed AST. The structured path reads attributes directly and, unlike the text path's extract_hcl_attr, performed no interpolation, surfacing values such as${var.vm_size}as machine types.Resolve asset identifiers through one shared helper across all ten asset categories, so no name reaches a deliverable as raw expression syntax. Where a reference cannot be resolved statically, prefer the Terraform logical name over a stripped fragment such as "bucket", which is generic, collision-prone and indistinguishable from a real name.
Emit compute instance exposure as
has_public_ip. The structured path usedhas_pub_ip, so the POA&M rule for internet-exposed instances, which testshas_public_ip is True, could never fire for anything ingested structurally, including via the recommendedterraform show -jsonpath.Tolerate scalar
protocolandportsfirewall values. A single-element variable default such as[5432]is stored unwrapped, soports = var.xarrives as an int; iterating it raised TypeError and aborted extraction for an entire blueprint rather than degrading on one field.State an unresolvable CMEK reference in words instead of stripping
${var.kms_key_name}down tokms_key_name, which reads as a real key name in the SSP.Make two HCL parser tests backend-aware. They encoded the in-repo fallback parser's behaviour and passed only when python-hcl2 was absent.
Verification
Measured across all 43 leaf blueprints: 43/43 extract with no failures and no HCL parse gaps; asset names containing unresolved expressions drop from 2 to 0 while the inventory retains all 52 asset items. Engine suite is 386/386, with 10 new regression tests covering CMEK key survival, boundary completeness for expression-named resources, expression-free identifiers, variable resolution, the POA&M field name and firewall shape tolerance. A full extract/generate/validate run against blueprints/il5/bigquery produces 71 schema-valid deliverables with 0 unresolved tokens.
Known limitation
Module-created infrastructure is not visible to static HCL scanning, so blueprints whose resources live behind
moduleblocks report few or no assets. Driving the skill fromterraform show -jsonoutput captures them, arriving through the same structured path fixed here.Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
Fixes # (GitHub issue id)
Type of Change
Deployment & Compliance Impact
Checklist
Code Quality & Reusability
modules/orfast/can be leveraged for this change.documentation/naming-convention.md.Documentation
README.mdof the modified module or blueprint.Security
Testing
Testing Performed
No local testing performed