Add per-template Interactsh eviction timeout - #7581
Conversation
WalkthroughChangesInteractsh eviction configuration
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Template
participant ProtocolRequest
participant Interactsh
participant RequestCache
Template->>ProtocolRequest: provide InteractshEviction
ProtocolRequest->>Interactsh: submit RequestData with Eviction duration
Interactsh->>RequestCache: store request event with selected TTL
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
pkg/templates/interactsh_eviction_test.go (1)
10-27: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover compile-time propagation, not only YAML decoding.
This test would still pass if
parseTemplateNoVerifystopped copying the value intotemplate.Options. Add an assertion through the real parse/compile path forTemplate.Options.InteractshEvictionorInteractshEvictionDuration().🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/templates/interactsh_eviction_test.go` around lines 10 - 27, The test currently verifies only YAML unmarshalling and must also cover propagation through the real template parse/compile path. Extend TestTemplateInteractshEvictionUnmarshal to parse or compile the raw template via parseTemplateNoVerify, then assert the resulting Template.Options.InteractshEviction or InteractshEvictionDuration() reflects 300 while retaining the existing unmarshal assertion.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pkg/templates/compile.go`:
- Line 558: Validate Template.InteractshEviction before assigning it to
options.InteractshEviction in the compile flow: reject values less than 0, while
preserving 0 as the unset/default state and allowing positive values. Ensure
invalid configurations return an error instead of reaching EvictionFromSeconds
and being converted to the global TTL.
---
Nitpick comments:
In `@pkg/templates/interactsh_eviction_test.go`:
- Around line 10-27: The test currently verifies only YAML unmarshalling and
must also cover propagation through the real template parse/compile path. Extend
TestTemplateInteractshEvictionUnmarshal to parse or compile the raw template via
parseTemplateNoVerify, then assert the resulting
Template.Options.InteractshEviction or InteractshEvictionDuration() reflects 300
while retaining the existing unmarshal assertion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4a0772e0-fa55-4ac5-8cb1-2f846984c049
📒 Files selected for processing (13)
pkg/protocols/code/code.gopkg/protocols/common/interactsh/eviction_test.gopkg/protocols/common/interactsh/interactsh.gopkg/protocols/headless/request.gopkg/protocols/http/request.gopkg/protocols/http/request_fuzz.gopkg/protocols/interactsh_eviction_test.gopkg/protocols/javascript/js.gopkg/protocols/network/request.gopkg/protocols/protocols.gopkg/templates/compile.gopkg/templates/interactsh_eviction_test.gopkg/templates/templates.go
| options.TemplateID = template.ID | ||
| options.TemplateInfo = template.Info | ||
| options.StopAtFirstMatch = template.StopAtFirstMatch | ||
| options.InteractshEviction = template.InteractshEviction |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files:"
fd -a 'compile.go|eviction|InteractshEviction' . | sed 's#^\./##' | head -100
echo
echo "Relevant occurrences:"
rg -n "InteractshEviction|EvictionFromSeconds|minimum=1|parseTemplateNoVerify" . | head -200
echo
echo "compile.go context around 530-575:"
if [ -f pkg/templates/compile.go ]; then
sed -n '530,575p' pkg/templates/compile.go | nl -ba -v530
fiRepository: projectdiscovery/nuclei
Length of output: 4297
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "pkg/templates/compile.go relevant snippets:"
awk 'NR>=460 && NR<=568 {printf "%6d\t%s\n", NR, $0}' pkg/templates/compile.go
echo
echo "pkg/templates/templates.go InteractshEviction declarations:"
awk 'NR>=125 && NR<=145 {printf "%6d\t%s\n", NR, $0}' pkg/templates/templates.go
echo
echo "pkg/protocols/protocols.go InteractshEviction declarations/helpers:"
awk 'NR>=100 && NR<=118 {printf "%6d\t%s\n", NR, $0}' pkg/protocols/protocols.go
awk 'NR>=310 && NR<=358 {printf "%6d\t%s\n", NR, $0}' pkg/protocols/protocols.go
echo
echo "pkg/protocols/common/interactsh/interactsh.go EvictionFromSeconds:"
awk 'NR>=380 && NR<=398 {printf "%6d\t%s\n", NR, $0}' pkg/protocols/common/interactsh/interactsh.go
echo
echo "pkg/templates/interactsh_eviction_test.go:"
awk '{printf "%6d\t%s\n", NR, $0}' pkg/templates/interactsh_eviction_test.go
echo
echo "validation search for InteractshEviction:"
rg -n "InteractshEviction|Interactsh.*Eviction|eviction.*minimum|min.*eviction|minimum=1" pkg cmd internal 2>/dev/null | head -200
echo
echo "JSON schema validation setup:"
rg -n "jsonschema|Validate|jsonschema\.|JSONSchema" pkg/protocols pkg/templates pkg/operators 2>/dev/null | head -200Repository: projectdiscovery/nuclei
Length of output: 50379
🏁 Script executed:
#!/bin/bash
set -EUo pipefail
echo "Template validation call sites:"
awk 'NR>=300 && NR<=350 {printf "%6d\t%s\n", NR, $0}' pkg/templates/templates.go
awk 'NR>=540 && PRIN 580 {printf "%6d\t%s\n", NR, $0}' pkg/templates/templates.go 2>/dev/null || true
rg -n "func .*Validate|tplValidator|Validate|jsonschema" pkg/templates/templates.go pkg/templates/parser.go pkg/templates/*.go 2>/dev/null | head -120
echo
echo "Behavioral probe: YAML unmarshals non-positive eviction fields and InteractshEvictionDuration delegates conversion:"
python3 - <<'PY'
from pathlib import Path
import re
templates_go = Path("pkg/templates/templates.go").read_text()
protocols_go = Path("pkg/protocols/protocols.go").read_text()
compile_go = Path("pkg/templates/compile.go").read_text()
interactsh_go = Path("pkg/protocols/common/interactsh/interactsh.go").read_text()
field = re.search(r'InteractshEviction\s+int`.*`', templates_go).group(0)
jsonschema_tag = re.search(r'jsonschema:"([^"]*)"', field)
print("field_tag:", jsonschema_tag.group(0) if jsonschema_tag else "NO_TAG")
print("field_has_minimum_tag:", "minimum=1" in (jsonschema_tag.group(1) if jsonschema_tag else ""))
print("field_has_validate_tag:", bool(re.search(r'validate:.*[<>!=]', field)))
print("compiles_at_line_558:", "options.InteractshEviction = template.InteractshEviction" in compile_go)
func = re.search(r'func \(e \*ExecutorOptions\) InteractshEvictionDuration\(\).*?return.*?time\.Duration\([^)]*\)\*', protocols_go, re.S).group(0)
print("duration_calls_EvictionFromSeconds:", "InteractshEviction" in func and "interactsh.EvictionFromSeconds(e.InteractshEviction)" in protocols_go)
print("EvictionFromSeconds_negative_returns_zero:", bool(re.search(r'func EvictionFromSeconds\(seconds int\).*?if seconds <= 0 \{\s*return 0\s*\}', interactsh_go, re.S)))
# Inspect applyTemplateVerification body and whether it validates Template.InteractshEviction explicitly.
body = re.search(r'func applyTemplateVerification\(template \*Template, data \[\]byte\).+?\n', "pkg/templates/templates.go", re.S)
print("has_applyTemplateVerification_function:", True)
PYRepository: projectdiscovery/nuclei
Length of output: 20946
🌐 Web query:
go-playground go yaml unmarshal jsonschema minimum validation required custom tag
💡 Result:
To implement JSON Schema validation for a struct in Go—especially one populated from YAML—you generally need to separate the unmarshaling phase from the validation phase, as the standard go-playground/validator package uses struct tags for validation rather than JSON Schema [1][2][3]. Since Go lacks a native JSON Schema layer, you have three primary approaches to achieve this requirement: 1. Use a Dedicated JSON Schema Library Libraries such as kaptinlin/jsonschema or go-rotini/jsonschema are designed specifically to validate data against a schema [4][5][6]. This is the most robust approach for JSON Schema compliance. - Workflow: First, unmarshal your YAML into a map or struct, then pass it to the library's validation method (e.g., schema.Validate(data)) before proceeding with application logic [4][5][7]. - Some libraries allow you to register default functions or perform "smart unmarshaling" that applies schema defaults [5][6][7]. 2. Use a Custom Validator with go-playground/validator You can implement a custom validation function in the go-playground/validator package that performs JSON Schema checks [2][8]. - Workflow: Register a custom validation tag (e.g., jsonschema) using v.RegisterValidation("jsonschema", customFunc) [2][8]. Inside this function, use a JSON Schema validator library (like gojsonschema) to validate the field's content against your schema [8]. - Note: This adds a dependency and requires manual management of your schema definitions within your Go code [8]. 3. Dedicated YAML Validation Libraries If your requirement is specifically to validate YAML structure and content, libraries like go-yamlvalidator allow you to define schemas (in YAML or JSON) and enforce rules like required fields, type checks, and conditional logic directly on YAML input [9]. Summary of Key Differences: - Structural Tags: The standard validator tags (e.g., validate:"required", validate:"min=10") are highly optimized for direct struct-field validation but do not natively understand complex JSON Schema objects [1][2][10]. - Workflow Separation: To use JSON Schema, you must move away from the "validate-on-unmarshal" pattern and adopt a "unmarshal then validate" pattern [4][7]. This prevents the overhead of validation from blocking the initial data decoding and allows you to use rich JSON Schema features that go-playground/validator does not support [5][7]. Recommendation: For production environments requiring full JSON Schema support, adopt a specialized library like kaptinlin/jsonschema, which handles both validation and schema-aware unmarshaling, keeping your validation logic clean and externalized from struct tags [5][6][7].
Citations:
- 1: https://github.com/go-playground/validator/blob/master/README.md
- 2: https://github.com/go-playground/validator/blob/393130c3f42f8dec286661944974f75c7eb0c1bd/doc.go
- 3: https://github.com/go-playground/validator
- 4: https://github.com/go-rotini/jsonschema/blob/main/README.md
- 5: https://github.com/kaptinlin/jsonschema
- 6: https://pkg.go.dev/github.com/kaptinlin/jsonschema
- 7: https://github.com/kaptinlin/jsonschema/blob/main/readme.md
- 8: https://qiita.com/nownabe/items/e62bbedd2e1687d1ff70
- 9: https://pkg.go.dev/github.com/Yakwilik/go-yamlvalidator@v0.2.1
- 10: https://pkg.go.dev/github.com/go-playground/Validator
🌐 Web query:
invopop jsonschema jsonschema validator minimum 1 struct tag validation yaml tag optional
💡 Result:
The invopop/jsonschema library is primarily a JSON Schema generation tool that uses Go struct tags via reflection [1][2]. It does not provide built-in runtime validation of the JSON data against the generated schema; rather, it focuses on generating the schema definition itself from your Go types [2]. Regarding your specific inquiries about struct tag usage: 1. Minimum/Maximum Validation: You can use standard JSON Schema keywords within the jsonschema tag. For example, to set minimum and maximum constraints, use: type Example struct { Value int json:"value" jsonschema:"minimum=1,maximum=10" } [1][3] 2. Optional Fields: In invopop/jsonschema, fields are generally treated as optional if they are omitted from the required list in the generated schema [1][2]. To control this, you can use the standard Go json tag with omitempty: type Example struct { OptionalField string json:"optional_field,omitempty" } [1][3] The library automatically handles the mapping of Go types to JSON Schema. If you need explicit control over nullability, the library includes configurations like NullableFromJSONSchemaTags, which can be enabled to look for a jsonschema:"nullable" tag [4]. 3. YAML Tags: The library primarily respects the json tag for field naming and structure [1][2]. While it may not explicitly process yaml tags for field naming by default, you can configure the reflector to use different tags for field names by setting the FieldNameTag configuration (e.g., to "yaml") [5]. Since this library only generates schemas, you will need a separate validator package (such as santhosh-tekuri/jsonschema or others) to perform the actual validation of your JSON data against the schema produced by invopop/jsonschema [6].
Citations:
- 1: https://pkg.go.dev/github.com/invopop/jsonschema
- 2: https://github.com/invopop/jsonschema
- 3: https://github.com/invopop/jsonschema/blob/main/reflect_test.go
- 4: feat: Add
NullableFromJSONSchemaTagsoption invopop/jsonschema#106 - 5: Add FieldNameTag config invopop/jsonschema#64
- 6: https://github.com/tylergannon/go-gen-jsonschema
Reject invalid interactsh-eviction values before propagation.
jsonschema:"minimum=1" only affects schema generation; YAML/JSON unmarshalling still stores values like -1 in Template.InteractshEviction. That value then reaches line 558, and EvictionFromSeconds converts it to 0, silently using the global TTL instead of rejecting the invalid template config. Reject values < 0 here while keeping 0 as the unset/default state.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pkg/templates/compile.go` at line 558, Validate Template.InteractshEviction
before assigning it to options.InteractshEviction in the compile flow: reject
values less than 0, while preserving 0 as the unset/default state and allowing
positive values. Ensure invalid configurations return an error instead of
reaching EvictionFromSeconds and being converted to the global TTL.
1 participant
Summary
interactsh-eviction(seconds) on templates so OAST request cache TTL can be longer for slow callbacks without raising the global-interactions-evictionfor every templateRequestEventcache expiry; unset keeps the global defaultCloses #5061
Summary by CodeRabbit
New Features
interactsh-evictionsetting to control how long pending interaction requests are retained.Bug Fixes