Repository navigation
test: check the Rego policies the docs tell people to copy - #433
Merged
Merged
Conversation
A reader copies these blocks. One that does not compile wastes their time.
One that compiles but denies everything is worse, because it looks like it
worked.
The test holds a block to that bar when it is presented as a whole file:
either it declares its own package, or its fence names a .rego file, as in
```rego pr-compliant.rego. Name the file whenever the page tells someone to
create it, and this holds the block to compiling from then on.
Blocks with neither are fragments showing one rule. They cannot compile
alone because their inputs are unbound, so there is nothing to check.
`default allow := false` on its own does not satisfy the allow check. A
policy can carry that line, have its real rule misspelled, compile, and
deny every input forever. That is the failure this exists to catch, so the
check wants a rule that can actually become true.
It found one. tutorials/linking_trails_across_branches.md had three faults
in nine lines: `package main`, which Kosli never reads because it queries
data.policy.allow; pre-1.0 syntax that no longer compiles; and
input.trail.compliance, where the field is input.trail.compliance_status.
Fixed here.
CI installs opa, pinned to v1.9.0 and checked against its published
sha256, matching how this workflow pins its actions. Without opa the
compile checks skip locally but fail outright when CI is set, so the suite
cannot go quietly green.
verified: pytest tests/ -> 58 passed. mint broken-links reports one break,
in tutorials/working_with_controls.mdx, pre-existing on main.
mutation: misspell the allow rule but keep the default line -> "defines no
allow rule that can be true". Drop the package from a named block
-> "is shown as a file to create but declares no package". Rename
the package -> "so Kosli never reads it". Break the syntax -> opa's
rego_parse_error. Each fails by name. A fence carrying attributes
rather than a filename stays a fragment.
search: counted every fenced block on the site, 881 of them. bash, shell,
yaml and json were measured and left out: their failures are
placeholders like <your-org>, CloudFormation !Ref and `...`
ellipses, so a check there reports conventions, not defects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
Contributor
|
Claude finished @AlexKantor87's task in 2m 28s —— View job Follow-up review:
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
The Rego fix in the previous commit corrected input.trail.compliance to
compliance_status. The same wrong field is read twice more on the same
page, in the jq tab, and I did not look.
`kosli get trail --output json` has no compliance key. So jq printed null
and the next step ran with --compliant="null".
verified: extracted the documented get trail sample from
client_reference/kosli_get_trail.md and ran both expressions
against it. .compliance.is_compliant gives null,
.compliance_status.is_compliant gives true.
A top-level is_compliant exists too, so .is_compliant would work. The
nested path matches the sentence above the block and the Rego block on the
same page.
search: grep -rnE "\.compliance\." over every md and mdx -> these two lines
and nothing else. Same for jq reads of trail JSON.
no mutation: prose and shell, no test. pytest tests/ still 58 passed.
Raised by the review bot on #433.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AlexKantor87
enabled auto-merge (squash)
September 28, 2026 13:44
mbevc1
reviewed
Sep 28, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An empty parametrize list is 0 tests and exit 0, so the whole check disappeared silently if the fence pattern stopped matching. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
Done - using the official action now, pinned to the v2.4.0 commit. Agreed on upstream, I won't open that from here. Also in this push: my own test could pass while testing nothing. It runs over a list of Rego blocks found by a pattern; if the pattern stops matching, the list is empty, pytest runs 0 tests and exits 0. Added a check that the list isn't empty. search: grep -rnE 'curl .*(-o|--output)' .github/workflows/ |
mbevc1
approved these changes
Sep 29, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Readers copy the Rego blocks on this site. One that does not compile wastes their time. One that compiles but denies everything is worse, because it looks like it worked.
This PR fixes some of the problematic rego.