httpapi: validate /api/concurrency payloads to prevent invalid settings; add tests - #7623
Open
eltypical wants to merge 58 commits into
Open
httpapi: validate /api/concurrency payloads to prevent invalid settings; add tests#7623eltypical wants to merge 58 commits into
eltypical wants to merge 58 commits into
Conversation
…7528) Bumps the modules group with 5 updates: | Package | From | To | | --- | --- | --- | | [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) | `0.5.11` | `0.5.12` | | [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.16` | `1.3.17` | | [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.41` | `0.1.42` | | [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.87` | `0.2.88` | | [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.42` | `1.2.43` | Updates `github.com/projectdiscovery/fastdialer` from 0.5.11 to 0.5.12 - [Release notes](https://github.com/projectdiscovery/fastdialer/releases) - [Commits](projectdiscovery/fastdialer@v0.5.11...v0.5.12) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.16 to 1.3.17 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.16...v1.3.17) Updates `github.com/projectdiscovery/networkpolicy` from 0.1.41 to 0.1.42 - [Release notes](https://github.com/projectdiscovery/networkpolicy/releases) - [Commits](projectdiscovery/networkpolicy@v0.1.41...v0.1.42) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.87 to 0.2.88 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.87...v0.2.88) Updates `github.com/projectdiscovery/cdncheck` from 1.2.42 to 1.2.43 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.42...v1.2.43) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/fastdialer dependency-version: 0.5.12 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.17 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/networkpolicy dependency-version: 0.1.42 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.88 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.43 dependency-type: indirect update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* grpc library * stabilize test
…7532) Bumps the modules group with 5 updates: | Package | From | To | | --- | --- | --- | | [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) | `0.5.12` | `0.5.13` | | [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.17` | `1.3.18` | | [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) | `1.9.0` | `1.10.0` | | [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.88` | `0.2.89` | | [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.43` | `1.2.44` | Updates `github.com/projectdiscovery/fastdialer` from 0.5.12 to 0.5.13 - [Release notes](https://github.com/projectdiscovery/fastdialer/releases) - [Commits](projectdiscovery/fastdialer@v0.5.12...v0.5.13) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.17 to 1.3.18 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.17...v1.3.18) Updates `github.com/projectdiscovery/httpx` from 1.9.0 to 1.10.0 - [Release notes](https://github.com/projectdiscovery/httpx/releases) - [Commits](projectdiscovery/httpx@v1.9.0...v1.10.0) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.88 to 0.2.89 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.88...v0.2.89) Updates `github.com/projectdiscovery/cdncheck` from 1.2.43 to 1.2.44 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.43...v1.2.44) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/fastdialer dependency-version: 0.5.13 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.18 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/httpx dependency-version: 1.10.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.89 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.44 dependency-type: indirect update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…tes (projectdiscovery#7533) Bumps the go_modules group with 1 update in the / directory: [google.golang.org/grpc](https://github.com/grpc/grpc-go). Updates `google.golang.org/grpc` from 1.72.2 to 1.79.3 - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.72.2...v1.79.3) Updates `github.com/go-jose/go-jose/v4` from 4.0.5 to 4.1.3 - [Release notes](https://github.com/go-jose/go-jose/releases) - [Commits](go-jose/go-jose@v4.0.5...v4.1.3) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-version: 1.79.3 dependency-type: direct:production dependency-group: go_modules - dependency-name: github.com/go-jose/go-jose/v4 dependency-version: 4.1.3 dependency-type: indirect dependency-group: go_modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ry#7537) Build the signer fragment from `PublicKey.Bytes()` instead of reading the raw ECDSA X coordinate. Keep the existing fragment format by hashing the trimmed X-coordinate bytes, and propagate key encoding errors thru signing & verification. Signed-off-by: Dwi Siswanto <git@dw1.io>
…ry#7549) Render locator arguments thru the action renderer for every element lookup mode (selector, regex, XPath, JavaScript, and search locators). Keep rendered values local and reuse selectors instead of rendering them twice. Render `waitvisible` locators on the owning page so Interactsh URLs remain attached to the request. Use a local URL source in tests to avoid network access. Closes projectdiscovery#7542 Signed-off-by: Dwi Siswanto <git@dw1.io>
…of-range panic (projectdiscovery#7531) * Guard against a negative regex extractor group to avoid an index-out-of-range panic A community template with group: -1 crashes the scan since the protocol path has no recover. ExtractRegex indexes match[e.RegexGroup], and the len(match) < groupPlusOne guard never catches a negative group, so match[-1] panics with index out of range [-1]. A negative group is meaningless, so return no results early instead. Signed-off-by: Arpit Jain <arpitjain099@gmail.com> * Harden extractors --------- Signed-off-by: Arpit Jain <arpitjain099@gmail.com> Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
…overy#7515) Co-authored-by: james-yusuke <ycpl@yecov.com>
Bumps the workflows group with 1 update in the / directory: [crate-ci/typos](https://github.com/crate-ci/typos). Updates `crate-ci/typos` from 1.47.2 to 1.48.0 - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.47.2...v1.48.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-version: 1.48.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: workflows ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the go_modules group with 1 update in the / directory: [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose). Updates `github.com/go-jose/go-jose/v4` from 4.1.3 to 4.1.4 - [Release notes](https://github.com/go-jose/go-jose/releases) - [Commits](go-jose/go-jose@v4.1.3...v4.1.4) --- updated-dependencies: - dependency-name: github.com/go-jose/go-jose/v4 dependency-version: 4.1.4 dependency-type: indirect dependency-group: go_modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…7554) Bumps the modules group with 3 updates: [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go), [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) and [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck). Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.18 to 1.3.19 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.18...v1.3.19) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.89 to 0.2.90 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.89...v0.2.90) Updates `github.com/projectdiscovery/cdncheck` from 1.2.44 to 1.2.45 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.44...v1.2.45) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.19 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.90 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.45 dependency-type: indirect update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…rojectdiscovery#7523) MatchWords lowercased the corpus and literal words but not the rendered template word, so a case-insensitive word matcher backed by a {{var}} that resolved to an uppercase value never matched (false negative). Lowercase the rendered word when CaseInsensitive is set. Co-authored-by: Synvoya <16019863+Synvoya@users.noreply.github.com>
* fix(monitor): stop busy-spinning hang monitor * test(integration): serialize dsl matcher variable case * drop serial --------- Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
…y#7428) * feat(protocols): add duration fields to other events Add matcher and extractor duration data[1] for DNS, SSL, WHOIS, TCP, WebSocket, and Headless requests. Track step durations for TCP inputs, WebSocket message exchanges, and Headless actions. The `duration` field holds the most recently measured operation. Expose request ID-prefixed duration values through the existing template context path, and add protocol tests for duration extraction, aliases, and request ID handling. [1]: The data exposed is `duration`, `duration_N`, `<request-id>_duration`, and `<request-id>_duration_<N>`. Closes projectdiscovery#5413 Closes projectdiscovery#7422 Signed-off-by: Dwi Siswanto <git@dw1.io> * test(protocols): added deterministic 10ms test-server delays Signed-off-by: Dwi Siswanto <git@dw1.io> * test: added a shared integration-test delay constant Signed-off-by: Dwi Siswanto <git@dw1.io> --------- Signed-off-by: Dwi Siswanto <git@dw1.io> Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
…tdiscovery#7278) * refactor: replace Split in loops with more efficient SplitSeq Signed-off-by: stringsbuilder <stringsbuilder@outlook.com> * minor refactor * strict checksum * fix typo * harden sdk tests Avoid shared templates-dir update races across parallel go test packages and ignore known leveldb/ratelimit/memguardian goroutines in goleak. * bind splitseq temps Keep SplitSeq inputs in locals so iterator substrings cannot outlive a temporary string expression. * fix sdk templates Keep goleak ignores, drop DisableUpdateCheck so CI can install templates, and serialize UpdateIfOutdated across processes. --------- Signed-off-by: stringsbuilder <stringsbuilder@outlook.com> Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* improving lua script with args and values * fixing comments * keys to string * fixing context * fix(test): drop log() calls inflating redis-lua-script result count The two log() calls each emitted a [JS] stdout line that the integration harness counted as a result, yielding 3 results vs the expected 1. The matcher reads `response` from the final expression value (not stdout), so returning the get RunLuaScript call directly keeps the assertion intact with a single result. * fix flaky * harden sdk tests Avoid shared templates-dir update races across parallel go test packages and ignore known leveldb/ratelimit/memguardian goroutines in goleak. * fix sdk templates Keep goleak ignores, drop DisableUpdateCheck so CI can install templates, and serialize UpdateIfOutdated across processes.
…covery#7536) * fix(index): limit include-tags to matching tag exclusions IncludeTags in the metadata index was treated as a forced include, so `-itags` could bypass explicit ID/path filters and other include criteria, while also let a template thru when it had both an included tag and a different excluded tag. Match the parsed-template filter instead: an included tag only cancels exclusion for that same tag. Other exclusions and normal include filters still apply, while IncludeTemplates remains the explicit path override. Fixes projectdiscovery#7534 Signed-off-by: Dwi Siswanto <git@dw1.io> * Skip include-tags functional parity until release --------- Signed-off-by: Dwi Siswanto <git@dw1.io> Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Bumps the go_modules group with 1 update in the / directory: [google.golang.org/grpc](https://github.com/grpc/grpc-go). Updates `google.golang.org/grpc` from 1.79.3 to 1.82.1 - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.79.3...v1.82.1) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-version: 1.82.1 dependency-type: direct:production dependency-group: go_modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…dk-result-callback sdk: per-execution result callback for thread-safe engine
Signed-off-by: Dwi Siswanto <git@dw1.io>
Signed-off-by: Dwi Siswanto <git@dw1.io>
* max time * goflags bump
Bumps the go_modules group with 1 update in the / directory: [github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi). Updates `github.com/getkin/kin-openapi` from 0.132.0 to 0.144.0 - [Release notes](https://github.com/getkin/kin-openapi/releases) - [Commits](getkin/kin-openapi@v0.132.0...v0.144.0) --- updated-dependencies: - dependency-name: github.com/getkin/kin-openapi dependency-version: 0.144.0 dependency-type: direct:production dependency-group: go_modules ... Signed-off-by: dependabot[bot] <support@github.com>
kin-openapi now represents exclusive minimum and maximum as `ExclusiveBound` values. Use `IsTrue()` to preserve their boolean semantics when generating numeric examples. Signed-off-by: Dwi Siswanto <git@dw1.io>
…ttp-tls-metadata http tlsmeta
…abot/go_modules/go_modules-715adc688b chore(deps): bump github.com/getkin/kin-openapi from 0.132.0 to 0.144.0 in the go_modules group across 1 directory
…want0/feat/lib/reuse-metadata-cache-across-thread-safe-scans feat(lib): reuse metadata cache across thread-safe scans
…t-nil-request fix: avoid panic in BuildRequest when request response has no request
…li-leaktest Add CLI runner goroutine leak test
* extend smb * enum services * unify session * bridge files * document matrix * enum processes * test session * test enums * test bridge * fix ci * address review * fix lint
…7615) Bumps the modules group with 5 updates: | Package | From | To | | --- | --- | --- | | [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) | `0.1.90` | `0.1.91` | | [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.19` | `1.3.20` | | [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.42` | `0.1.43` | | [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.90` | `0.2.91` | | [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.45` | `1.2.46` | Updates `github.com/projectdiscovery/rawhttp` from 0.1.90 to 0.1.91 - [Release notes](https://github.com/projectdiscovery/rawhttp/releases) - [Commits](projectdiscovery/rawhttp@v0.1.90...v0.1.91) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.19 to 1.3.20 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.19...v1.3.20) Updates `github.com/projectdiscovery/networkpolicy` from 0.1.42 to 0.1.43 - [Release notes](https://github.com/projectdiscovery/networkpolicy/releases) - [Commits](projectdiscovery/networkpolicy@v0.1.42...v0.1.43) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.90 to 0.2.91 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.90...v0.2.91) Updates `github.com/projectdiscovery/cdncheck` from 1.2.45 to 1.2.46 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.45...v1.2.46) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/rawhttp dependency-version: 0.1.91 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.20 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/networkpolicy dependency-version: 0.1.43 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.91 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.46 dependency-type: indirect update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…scovery#7575) * Native fingerprint * fix typo
* mssql fingerprint * review fixes * fix typo * fix test
Do not let parsed template cache hits bypass protocol compilation, making `-validate` reject templates with runtime compilation errors. Fixes projectdiscovery#7602 Signed-off-by: Dwi Siswanto <git@dw1.io>
* add http * harden ntlm * fix bounds
…want0/fix/loader/compile-requests-during-validation fix(loader): compile requests during validation
…lues - Enforce non-negative values for rate limit, bulk size, threads, payload/probe/javascript concurrency - Honor explicit 0 to reset concurrency settings to defaults - Validate rate_limit_duration format and bounds This prevents nil-pointer dereferences or unintended overflows when invalid values are posted to /api/concurrency. Signed-off-by: Kiell Tampubolon <113831023+eltypical@users.noreply.github.com>
Covers negative values rejection and duration parsing with non-negative bound. Ensures 0 is accepted but does not change settings. Signed-off-by: Kiell Tampubolon <113831023+eltypical@users.noreply.github.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add input validation to internal/httpapi API endpoint that adjusts runtime concurrency. Reject negative values and invalid or negative rate_limit_duration, and accept 0 as a no-op reset. This prevents invalid values from propagating into runner options and the JS engine concurrency metrics.
Root-cause trace
Impact Matrix
Tests
How to run
Notes
Signed-off-by: Kiell Tampubolon 113831023+eltypical@users.noreply.github.com