Skip to content

Add sandboxed file() DSL helper for static payloads - #7583

Open
Mzack9999 wants to merge 8 commits into
devfrom
5402-file-helper
Open

Add sandboxed file() DSL helper for static payloads#7583
Mzack9999 wants to merge 8 commits into
devfrom
5402-file-helper

Conversation

@Mzack9999

@Mzack9999 Mzack9999 commented Jul 24, 2026

Copy link
Copy Markdown
Member

Summary

  • Add file(path) DSL helper that returns whole-file contents (including binary) for static payloads
  • Reuse LoadHelperFile sandbox (templates dir, template-relative paths, -lfa, hard-link checks) with size and regular-file guards
  • Install per-goroutine template context during compile/execute so evaluation cannot escape the sandbox

Closes #5402

Summary by CodeRabbit

  • New Features
    • Added a file() helper to safely load file contents during template evaluation, using sandboxed, template-relative resolution and per-execution caching.
    • Added a goid() helper for DSL use.
    • Improved file() detection to trigger lazy evaluation only when an actual helper call is present.
  • Security
    • Enforces strict sandbox boundaries (blocks absolute paths, traversal, symlinks/hard links, directories, and non-regular files like FIFO) and caps reads to 10 MB; local file access remains optionally configurable.
  • Bug Fixes
    • Ensures the required file-loading sandbox context is consistently applied across template compilation/execution entry points.
  • Tests
    • Added extensive correctness, caching-isolation, nested-context, and size/argument validation coverage.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds a sandboxed file() DSL helper for binary-safe payload loading, size limits, caching, and goroutine-scoped context. Template execution installs the context, while variables containing file() are deferred until runtime. Tests cover loading, security, and concurrency.

Changes

File helper execution

Layer / File(s) Summary
File helper runtime
pkg/operators/common/dsl/*
Registers file(), loads files through the helper loader, enforces sandbox and size rules, caches contents, and isolates contexts by goroutine.
Executor context wiring
pkg/tmplexec/exec.go
Installs file-loading context during compilation and execution using executor metadata.
Lazy variable handling
pkg/protocols/common/variables/*
Detects actual file() calls and marks variables for lazy evaluation.
File helper validation
pkg/operators/common/dsl/file_test.go, pkg/operators/common/dsl/file_security_test.go, pkg/operators/common/dsl/file_security_unix_test.go
Tests path resolution, binary content, sandbox restrictions, file limits, caching, nested contexts, goroutine isolation, and FIFO rejection.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TemplateExecuter
  participant WithFileLoadContext
  participant file
  participant Options.LoadHelperFile
  TemplateExecuter->>WithFileLoadContext: Install executor file context
  WithFileLoadContext->>file: Evaluate file(path)
  file->>Options.LoadHelperFile: Load sandboxed helper file
  Options.LoadHelperFile-->>file: Return validated reader
  file-->>TemplateExecuter: Return file contents
Loading

Possibly related PRs

Poem

I’m a rabbit with bytes in my den,
Loading payloads again and again.
Sandbox doors firmly shut,
Cache tucked under my foot,
Binary treasures hop safely within.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 6.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding a sandboxed file() DSL helper for static payloads.
Linked Issues check ✅ Passed The PR implements a file helper that returns file contents for static payloads, including binary data, matching issue #5402.
Out of Scope Changes check ✅ Passed The changes are focused on implementing and integrating the file() helper, with supporting tests and sandbox plumbing only.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 5402-file-helper

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
pkg/operators/common/dsl/goid.go (1)

9-25: 🩺 Stability & Availability | 🔵 Trivial | ⚖️ Poor tradeoff

Goroutine-ID via runtime.Stack parsing is an unsupported Go trick.

goid() derives goroutine identity by parsing the text of runtime.Stack, which is documented for human-readable debug output only, not a stable public contract. If the trace format ever changes, goid() silently falls back to 0 for every goroutine, collapsing all concurrently-installed WithFileLoadContext sandboxes into one shared slot — a correctness/sandbox-isolation regression that would be hard to notice.

Consider using an actively-maintained library that already handles the fragile stack-parsing edge cases (e.g. petermattis/goid), or better, thread the FileLoadContext explicitly through the call chain (e.g., via context.Context) instead of relying on ambient goroutine-local state.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/operators/common/dsl/goid.go` around lines 9 - 25, Replace the
unsupported runtime.Stack parsing in goid with an actively maintained
goroutine-ID mechanism such as petermattis/goid, or refactor the file-load
sandbox flow to pass FileLoadContext explicitly through the call chain. Update
the surrounding WithFileLoadContext and file() integration so concurrent
sandboxes remain isolated and avoid silently sharing a zero-ID slot.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/protocols/common/variables/variables.go`:
- Around line 240-245: Update the file() detection in the variable analysis
logic to recognize file() only as a parsed DSL expression or through a
token-aware matcher, rather than using strings.Contains on the raw value.
Preserve lazy evaluation for genuine file() expressions while avoiding matches
inside ordinary text such as "profile(". Add a regression test covering literal
text containing file(.

In `@pkg/tmplexec/exec.go`:
- Around line 335-344: Update TemplateExecuter.fileLoadContext to lazily create
and retain a single *dsl.FileLoadContext per executor, using a synchronization
mechanism such as sync.Once for concurrent calls. Reuse that context across
Compile, Execute, and ExecuteWithResults invocations so its existing sync.Map
cache persists across targets, while preserving the nil behavior for an
uninitialized executer or options.

---

Nitpick comments:
In `@pkg/operators/common/dsl/goid.go`:
- Around line 9-25: Replace the unsupported runtime.Stack parsing in goid with
an actively maintained goroutine-ID mechanism such as petermattis/goid, or
refactor the file-load sandbox flow to pass FileLoadContext explicitly through
the call chain. Update the surrounding WithFileLoadContext and file()
integration so concurrent sandboxes remain isolated and avoid silently sharing a
zero-ID slot.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9be4db4f-065a-4d20-8be2-915ba1a64192

📥 Commits

Reviewing files that changed from the base of the PR and between bcf2089 and 8e389b3.

📒 Files selected for processing (7)
  • pkg/operators/common/dsl/dsl.go
  • pkg/operators/common/dsl/file.go
  • pkg/operators/common/dsl/file_test.go
  • pkg/operators/common/dsl/goid.go
  • pkg/protocols/common/variables/variables.go
  • pkg/protocols/common/variables/variables_test.go
  • pkg/tmplexec/exec.go

Comment thread pkg/protocols/common/variables/variables.go
Comment thread pkg/tmplexec/exec.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/operators/common/dsl/file_security_test.go (1)

65-75: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Minor: fold these two cases into the initial cases slice literal.

Repeating the anonymous struct{ name string; expr string } type just to append two more entries is unnecessary; they can be added directly to the slice literal at lines 45-63.

♻️ Proposed simplification
 		{name: "dot_path", expr: `file(".")`},
 		{name: "dotdot_path", expr: `file("..")`},
 		{name: "empty_components", expr: `file("////")`},
+		{name: "template_directory", expr: fileExpr(templateDir)},
+		{name: "templates_directory", expr: fileExpr(templatesDir)},
 	}
-
-	// Also deny reading the template directory itself and templates dir as paths.
-	cases = append(cases,
-		struct {
-			name string
-			expr string
-		}{name: "template_directory", expr: fileExpr(templateDir)},
-		struct {
-			name string
-			expr string
-		}{name: "templates_directory", expr: fileExpr(templatesDir)},
-	)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/operators/common/dsl/file_security_test.go` around lines 65 - 75, Update
the initial cases slice literal in the file-security test to include the
template_directory and templates_directory entries directly, and remove the
later append block and its repeated anonymous struct declarations.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/operators/common/dsl/file_security_test.go`:
- Around line 65-75: Update the initial cases slice literal in the file-security
test to include the template_directory and templates_directory entries directly,
and remove the later append block and its repeated anonymous struct
declarations.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b2b94c3f-266d-4fff-95f4-4c4876ddcb84

📥 Commits

Reviewing files that changed from the base of the PR and between 8e389b3 and 47d22c6.

📒 Files selected for processing (5)
  • pkg/operators/common/dsl/file_security_test.go
  • pkg/operators/common/dsl/file_test.go
  • pkg/protocols/common/variables/variables.go
  • pkg/protocols/common/variables/variables_test.go
  • pkg/tmplexec/exec.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • pkg/protocols/common/variables/variables_test.go
  • pkg/protocols/common/variables/variables.go
  • pkg/operators/common/dsl/file_test.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/operators/common/dsl/file_security_unix_test.go`:
- Around line 60-69: Strengthen the error assertion in the FIFO rejection test
to require the helper’s specific non-regular-file error or sentinel, rather than
accepting generic “could not load”, “denied”, or “device” messages. Retain only
error variants that are genuinely platform-specific while preserving the
existing require.Error check and errCh flow.
- Around line 39-58: Update the FIFO setup around writerCh and the file()
rejection check so observing the evaluator error does not depend on a blocking
O_WRONLY open completing. Use a non-blocking or RDWR FIFO opener, or select
concurrently between writerCh and errCh, while preserving cleanup for any
successfully opened writer and the existing rejection assertion.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8e7cd202-7fc6-4e00-a0c2-b8581289ac2f

📥 Commits

Reviewing files that changed from the base of the PR and between 47d22c6 and c2681f5.

📒 Files selected for processing (2)
  • pkg/operators/common/dsl/file_security_test.go
  • pkg/operators/common/dsl/file_security_unix_test.go
💤 Files with no reviewable changes (1)
  • pkg/operators/common/dsl/file_security_test.go

Comment thread pkg/operators/common/dsl/file_security_unix_test.go Outdated
Comment thread pkg/operators/common/dsl/file_security_unix_test.go Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add file helper function to help import static payloads from files

1 participant