feat: evaluate DSL helpers in -H values per request - #7618
Open
aviadavi wants to merge 3 commits into
Open
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
🚧 Files skipped from review as they are similar to previous changes (5)
WalkthroughThe DSL adds a non-cacheable ChangesDynamic custom header DSL
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant GeneratedRequest
participant setCustomHeaders
participant evaluateCustomHeaderExpressions
participant rand_user_agent
GeneratedRequest->>setCustomHeaders: apply custom headers
setCustomHeaders->>evaluateCustomHeaderExpressions: resolve embedded expressions
evaluateCustomHeaderExpressions->>rand_user_agent: evaluate helper
rand_user_agent-->>evaluateCustomHeaderExpressions: return random user-agent
evaluateCustomHeaderExpressions-->>setCustomHeaders: return resolved or raw fallback
setCustomHeaders-->>GeneratedRequest: set outgoing header
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pkg/protocols/http/request.go`:
- Around line 1302-1304: Update the custom-header handling around
evaluateCustomHeaderValue so raw requests preserve the literal header value and
only non-raw requests evaluate custom expressions; keep assigning the resulting
value to req.rawRequest.Headers for the raw path, and add a regression test
covering a raw request header such as -H '{{rand_user_agent()}}'.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: d9b651b9-8884-4fb3-8966-58451c5863bb
📒 Files selected for processing (4)
pkg/operators/common/dsl/dsl.gopkg/operators/common/dsl/rand_user_agent_test.gopkg/protocols/http/custom_headers_dsl_test.gopkg/protocols/http/request.go
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pkg/protocols/http/custom_headers_dsl_test.go`:
- Around line 72-88: Update TestUnsafeRawCustomHeaderIsEvaluated to exercise
production request generation through generateRawRequest and
TryFillCustomHeaders instead of calling evaluateCustomHeaderExpressions
directly. Inspect the generated raw request bytes and assert the DSL marker is
absent while the static “myprop/value” tag remains present; retain the
repeated-request check to confirm user-agent evaluation is not frozen.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3e2dfeb1-d659-429d-80dd-c6690cf7c84b
📒 Files selected for processing (3)
pkg/protocols/http/build_request.gopkg/protocols/http/custom_headers_dsl_test.gopkg/protocols/http/request.go
🚧 Files skipped from review as they are similar to previous changes (1)
- pkg/protocols/http/request.go
Adds a rand_user_agent() DSL helper and evaluates helper expressions in global custom header values once per generated request. This implements the approach proposed in projectdiscovery#7446 instead of a dedicated CLI flag, so attribution tagging needs no new global option: -H "User-Agent: {{rand_user_agent()}} myprop/value" The helper is registered with cacheable=false, otherwise a cached result would freeze the user agent for the whole run. It is backed by the existing projectdiscovery/useragent pool that nuclei already uses for UA randomization, so tagged scans send the same class of user agent as untagged ones and no new dependency is introduced. The name matches the slug dsl.FakerFunctions() would expose, so wiring faker later is a drop in. Values come from the operator's own -H flag or config file, the same trust boundary as a template, and response data is never evaluated here. Values with no {{ }} marker skip evaluation entirely, and compile or evaluation errors keep the original value so a typo degrades the header rather than failing the scan. The unsafe raw request path is unchanged.
Unsafe raw requests do not go through setCustomHeaders for the value that
reaches the wire: TryFillCustomHeaders splices the -H line straight into
UnsafeRawBytes, and rawhttp sends those bytes. So a header carrying an
expression was sent with the literal {{...}} marker intact.
Evaluate the -H strings before they are spliced in, and share one
evaluation helper between both paths so they cannot drift.
Verified against a local server with an unsafe: true template:
before: {{rand_user_agent()}} myprop/value
after: Mozilla/5.0 (Macintosh ...) Safari/605.1.15 myprop/value
The previous test called the resolver directly, so it passed even with the TryFillCustomHeaders wiring reverted and therefore guarded nothing. Drive real request generation instead and read UnsafeRawBytes, asserting the marker is absent, the tag survives, and the user agent still varies across requests. Confirmed by reverting the fix: the test now fails with "expression reached the wire unevaluated".
aviadavi
force-pushed
the
feat/dsl-in-custom-headers
branch
from
August 3, 2026 16:40
ae10190 to
0f5934c
Compare
Author
|
@dogancanbakir could you approve the workflow run here? CI passed on the first commit, but the pushes since then are sitting at Just rebased onto current No review needed from anyone yet, just the CI approval. Thanks. |
Author
|
@Mzack9999 sorry for the second ping. This is still waiting on a CI workflow approval, nothing else, and I asked the wrong person the first time. Happy to rebase again if that helps, but I held off since each push re-queues the runs and there are already a few sitting at |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #7617
What
rand_user_agent()DSL helper, backed byprojectdiscovery/useragent(already a direct dependency, already used for UA randomization). Registered withcacheable=false, since a cached result would freeze the UA for the whole run.setCustomHeadersnow resolves{{...}}in global header values. It already runs once per generated request, so this gives per-request evaluation with no new plumbing.33 lines of non-test code across 2 files.
Why this shape
Implements @dwisiswant0's suggestion from #7446 rather than the
-ua-tagflag from #7445 (closed). No new CLI flag, no new dependency.Note that
{{rand_ua}}as written in #7446 does not exist yet: nuclei wires onlydsl.HelperFunctions(), and the faker generators from projectdiscovery/dsl#234 sit in a separatedsl.FakerFunctions()that nuclei never merges. Hence piece 1. If you would rather mergeFakerFunctions()wholesale, I will swap it.Behaviour
{{skip evaluation entirely, so the common path is unaffected-Hflag or config file, the same trust boundary as a template. Response data is not evaluated on this path.UnsafeRawBytesviaTryFillCustomHeadersrather than fromsetCustomHeaders, so they are evaluated at the splicepoint instead; both paths share one evaluation helper so they cannot drift
Verification
Unit tests cover per-request rotation, tag retention, no-expression passthrough, broken-expression fallback, the non-cached helper, and argument rejection.
Live run, 4 requests against a local server with
-H "User-Agent: {{rand_user_agent()}} myprop/value":Four distinct UAs, tag on each.
go build ./...,go vet, andgo test ./pkg/protocols/http/...pass.Summary by CodeRabbit
rand_user_agent()helper for generating random user-agent values in expressions.