Skip to content

Add SSRF protection through HTTP policy validation - #1077

Open
cgalibern wants to merge 13 commits into
opensvc:mainfrom
cgalibern:dev
Open

Add SSRF protection through HTTP policy validation#1077
cgalibern wants to merge 13 commits into
opensvc:mainfrom
cgalibern:dev

Conversation

@cgalibern

@cgalibern cgalibern commented Aug 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

This Pull Request introduces Server-Side Request Forgery (SSRF) protection mechanisms, new utilities for handling HTTP policy validation, and improvements to error handling and code structuring in configuration handling.

Main Changes

  • Enhanced SSRF Protection:
    • Added new configuration options (allowed URLs, blocked URLs, CIDR ranges, and redirect settings) for SSRF defense in rawconfig/node.go and a new file rawconfig/ssrf_policy.go.
    • Implemented an HTTP policy checker (util/httppolicy) to validate URLs and IPs against SSRF rules.
    • Updated uri package (util/uri/uri.go) to enforce SSRF policies during URL fetching with logic for client customization.
  • Improvements in Error Handling:
    • Fixed potential silent failures in commoncmd/object_create.go by returning errors instead of nil in case of a fetch failure.
    • Enhanced type consistency in omcmd/object_create.go and oxcmd/object_create.go by converting methods to use pointers (*CmdObjectCreate).
  • Testing Additions:
    • Introduced comprehensive SSRF policy validation tests (util/httppolicy/main_test.go) to ensure proper functioning of the newly added HTTP policy utilities.
  • Minor Message Updates:
    • Updated help message in flags.go to better specify allowed URL protocols for configuration sources.

cgalibern added 13 commits August 12, 2026 09:54
@cgalibern
[oxcmd,omcmd] Use pointer receiver for CmdObjectCreate methods
7ff01ff 
Fix: Struct CmdObjectCreate has methods on both value and pointer
receivers
@cgalibern
[core/rawconfig] Add SSRF whitelist and blocked CIDR configuration ha…
8e15dd0 
…ndling

It defines defaults SSRFWhiteListUrl and SSRFBlockedCIDR from sysconfig like files for opensvc.
With following defaults when OSVC_SSRF_WHITELIST_URL and OSVC_SSRF_BLOCKED_CIDR are not defined.

    SSRFWhiteListUrl = []string{
            "https://raw.githubusercontent.com",
            "https://gitlab.com",
    }

    SSRFBlockedCIDR = []string{
            // RFC 1918 private address ranges
            "10.0.0.0/8",
            "172.16.0.0/12",
            "192.168.0.0/16",

            // RFC 4193 Unique Local IPv6 Unicast Addresses
            "fc00::/7",

            // Loopback ranges
            "127.0.0.0/8", // RFC 1122
            "::1/128",     // RFC 4291
    }
@cgalibern
[commoncmd] Fix error return in DataFromConfigURI when u.Fetch() …
c6066c6 
…fails
@cgalibern
[util/httppolicy] Add HTTP policy validation library and test suite
f1a0125 
Provide URL validation utility with whitelisting, CIDR blocking, and raw request handling.
Includes comprehensive test coverage.
@cgalibern
[util/uri] Add HTTP policy check to validate URLs against SSRF rules
e3e92a0
@cgalibern
[core/rawconfig] Define more flexible SSRF URL and CIDR configurations
7a7ae80 
Introduce SSRFAllowedURL, SSRFBlockedURL, SSRFAllowedCIDR, and
SSRFBlockedCIDR to replace the existing whitelist and blocked CIDR
configurations, improving configuration granularity and consistency
with new defaults.
@cgalibern
[core/rawconfig] Add SSRF redirect configuration support
beaa506 
Introduce `SSRFEnableRedirects` to allow configuration of redirect
handling in SSRF protections. Update `setSSRF` to include redirect
behavior based on `OSVC_SSRF_ENABLE_REDIRECTS` environment variable.

Default SSRFEnableRedirects value is false
@cgalibern
[util/httppolicy] Improves Check
aef2cde 
This commit improves the Check function with more granular rules:
- AllowedUrl: list of allowed urls
- BlockedUrl: list of blocking url patterns
- BlockedCIDR: list of blocked cidr
- AllowedCIDR: list of allowed cidr exceptions of the BlockedCIDR list
- Prevent TOCTOU bugs: the returned ip is the resolved ip that
  satisfies CIDR rules.

Updated tests:
    === RUN   TestCheck
    === RUN   TestCheck/raw_config_checker
        main_test.go:210: policy AllowedCIDR: [127.0.0.10/32 127.0.0.2/32 fd7a:115c:a1e0:ab12:4843:cd96:626b:430b/128]
        main_test.go:211: policy BlockedUrl: [*]
        main_test.go:212: policy BlockedCIDR: [10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 fc00::/7 127.0.0.0/8 ::1/128]
        main_test.go:213: policy AllowedCIDR: [127.0.0.10/32 127.0.0.2/32 fd7a:115c:a1e0:ab12:4843:cd96:626b:430b/128]
    === RUN   TestCheck/raw_config_checker/allows_https_github_relay_v3_config_path
        main_test.go:216: Check url: https://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf
        main_test.go:224: url detected ip 2606:50c0:8000::154, port 443
    --- PASS: TestCheck/raw_config_checker/allows_https_github_relay_v3_config_path (0.01s)
    === RUN   TestCheck/raw_config_checker/allows_https_github_host_only
        main_test.go:216: Check url: https://www.github.com
        main_test.go:224: url detected ip 140.82.121.4, port 443
    --- PASS: TestCheck/raw_config_checker/allows_https_github_host_only (0.00s)
    === RUN   TestCheck/raw_config_checker/allows_https_github_host_with_trailing_slash
        main_test.go:216: Check url: https://www.github.com/
        main_test.go:224: url detected ip 140.82.121.4, port 443
    --- PASS: TestCheck/raw_config_checker/allows_https_github_host_with_trailing_slash (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_http_scheme
        main_test.go:216: Check url: http://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf
        main_test.go:221: got expected error: reject url http://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf: url http://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_http_scheme (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_file_scheme_with_empty_host
        main_test.go:216: Check url: file:///tmp/foo
        main_test.go:221: got expected error: empty host
    --- PASS: TestCheck/raw_config_checker/rejects_file_scheme_with_empty_host (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_disallowed_port_on_allowlisted_host_path
        main_test.go:216: Check url: https://github.com:666/opensvc/om3
        main_test.go:221: got expected error: reject url https://github.com:666/opensvc/om3: url https://github.com:666/opensvc/om3 is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_disallowed_port_on_allowlisted_host_path (0.00s)
    === RUN   TestCheck/raw_config_checker/allows_exception_url_with_port_and_path
        main_test.go:216: Check url: https://github.com:8888/opensvc/om3
        main_test.go:224: url detected ip 140.82.121.4, port 8888
    --- PASS: TestCheck/raw_config_checker/allows_exception_url_with_port_and_path (0.00s)
    === RUN   TestCheck/raw_config_checker/allows_subpath_under_allowlisted_path
        main_test.go:216: Check url: https://github.com:8888/opensvc/om3/bar
        main_test.go:224: url detected ip 140.82.121.4, port 8888
    --- PASS: TestCheck/raw_config_checker/allows_subpath_under_allowlisted_path (0.00s)
    === RUN   TestCheck/raw_config_checker/allows_normalized_path_within_allowlisted_path
        main_test.go:216: Check url: https://github.com:8888/opensvc/om3/../om3/bar
        main_test.go:224: url detected ip 140.82.121.4, port 8888
    --- PASS: TestCheck/raw_config_checker/allows_normalized_path_within_allowlisted_path (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_normalized_path_outside_allowlisted_path
        main_test.go:216: Check url: https://github.com:8888/opensvc/om3/../oc3/bar
        main_test.go:221: got expected error: reject url https://github.com:8888/opensvc/om3/../oc3/bar: url https://github.com:8888/opensvc/oc3/bar is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_normalized_path_outside_allowlisted_path (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_escaped_dotdot_path_escape
        main_test.go:216: Check url: https://github.com:8888/opensvc/om3/%2e%2e/foo
        main_test.go:221: got expected error: reject url https://github.com:8888/opensvc/om3/../foo: url https://github.com:8888/opensvc/foo is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_escaped_dotdot_path_escape (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_path_prefix_without_separator
        main_test.go:216: Check url: https://github.com:8888/opensvc/om3foo
        main_test.go:221: got expected error: reject url https://github.com:8888/opensvc/om3foo: url https://github.com:8888/opensvc/om3foo is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_path_prefix_without_separator (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_path_outside_allowlisted_prefix
        main_test.go:216: Check url: https://github.com:8888/opensvc/foo/bar
        main_test.go:221: got expected error: reject url https://github.com:8888/opensvc/foo/bar: url https://github.com:8888/opensvc/foo/bar is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_path_outside_allowlisted_prefix (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_file_scheme_with_remote_style_host
        main_test.go:216: Check url: file://raw.githubusercontent.com/tmp/foo
        main_test.go:221: got expected error: reject url file://raw.githubusercontent.com/tmp/foo: url file://raw.githubusercontent.com/tmp/foo is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_file_scheme_with_remote_style_host (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_unknown_scheme
        main_test.go:216: Check url: foo://raw.githubusercontent.com
        main_test.go:221: got expected error: reject url foo://raw.githubusercontent.com: url foo://raw.githubusercontent.com/. is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_unknown_scheme (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_disallowed_port_on_allowlisted_raw_github_host
        main_test.go:216: Check url: https://raw.githubusercontent.com:8888
        main_test.go:221: got expected error: reject url https://raw.githubusercontent.com:8888: url https://raw.githubusercontent.com:8888/. is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_disallowed_port_on_allowlisted_raw_github_host (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_non_allowlisted_host
        main_test.go:216: Check url: https://google.com
        main_test.go:221: got expected error: reject url https://google.com: url https://google.com/. is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_non_allowlisted_host (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_ipv4_loopback_resolved_from_localhost
        main_test.go:216: Check url: https://localhost
        main_test.go:221: got expected error: reject host localhost ip 127.0.0.1: blocked net 127.0.0.0/8 and not in exception net list [127.0.0.10/32 127.0.0.2/32 fd7a:115c:a1e0:ab12:4843:cd96:626b:430b/128]
    --- PASS: TestCheck/raw_config_checker/rejects_ipv4_loopback_resolved_from_localhost (0.00s)
    === RUN   TestCheck/raw_config_checker/block_cidr
        main_test.go:216: Check url: https://127.0.0.3/foo
        main_test.go:221: got expected error: reject host 127.0.0.3 ip 127.0.0.3: blocked net 127.0.0.0/8 and not in exception net list [127.0.0.10/32 127.0.0.2/32 fd7a:115c:a1e0:ab12:4843:cd96:626b:430b/128]
    --- PASS: TestCheck/raw_config_checker/block_cidr (0.00s)
    === RUN   TestCheck/raw_config_checker/block_cidr_v4_mapped_v6
        main_test.go:216: Check url: https://[::ffff:127.0.0.3]/foo
        main_test.go:221: got expected error: reject host ::ffff:127.0.0.3 ip 127.0.0.3: blocked net 127.0.0.0/8 and not in exception net list [127.0.0.10/32 127.0.0.2/32 fd7a:115c:a1e0:ab12:4843:cd96:626b:430b/128]
    --- PASS: TestCheck/raw_config_checker/block_cidr_v4_mapped_v6 (0.00s)
    === RUN   TestCheck/raw_config_checker/allow_cidr_exception_v4_mapped_v6
        main_test.go:216: Check url: https://[::ffff:127.0.0.2]/foo
        main_test.go:224: url detected ip 127.0.0.2, port 443
    --- PASS: TestCheck/raw_config_checker/allow_cidr_exception_v4_mapped_v6 (0.00s)
    === RUN   TestCheck/raw_config_checker/allow_cidr_exception
        main_test.go:216: Check url: https://127.0.0.2/foo
        main_test.go:224: url detected ip 127.0.0.2, port 443
    --- PASS: TestCheck/raw_config_checker/allow_cidr_exception (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_ipv6_loopback
        main_test.go:216: Check url: https://[::1]
        main_test.go:221: got expected error: reject host ::1 ip ::1: blocked net ::1/128 and not in exception net list [127.0.0.10/32 127.0.0.2/32 fd7a:115c:a1e0:ab12:4843:cd96:626b:430b/128]
    --- PASS: TestCheck/raw_config_checker/rejects_ipv6_loopback (0.00s)
    === RUN   TestCheck/raw_config_checker/reject_ula_ipv6_cidr
        main_test.go:216: Check url: https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b]
        main_test.go:221: got expected error: reject host fd7a:115c:a1e0:ab12:4843:cd96:626b:626b ip fd7a:115c:a1e0:ab12:4843:cd96:626b:626b: blocked net fc00::/7 and not in exception net list [127.0.0.10/32 127.0.0.2/32 fd7a:115c:a1e0:ab12:4843:cd96:626b:430b/128]
    --- PASS: TestCheck/raw_config_checker/reject_ula_ipv6_cidr (0.00s)
    === RUN   TestCheck/raw_config_checker/allow_ula_ipv6_cidr_exception
        main_test.go:216: Check url: https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]
        main_test.go:224: url detected ip fd7a:115c:a1e0:ab12:4843:cd96:626b:430b, port 443
    --- PASS: TestCheck/raw_config_checker/allow_ula_ipv6_cidr_exception (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_private_ipv4_address
        main_test.go:216: Check url: https://10.0.0.1
        main_test.go:221: got expected error: reject url https://10.0.0.1: url https://10.0.0.1/. is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_private_ipv4_address (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_unique_local_ipv6_address
        main_test.go:216: Check url: https://[fc00::1]
        main_test.go:221: got expected error: reject url https://[fc00::1]: url https://[fc00::1]/. is blocked by pattern * and not in allowed list [https://raw.githubusercontent.com/opensvc/opensvc_templates https://8.8.8.8 https://github.com:8888/opensvc/om3 https://www.github.com https://localhost https://[::1] https://invalid-host-lookup.opensvc.com https://127.0.0.2/foo https://127.0.0.3/foo https://[::ffff:127.0.0.2]/foo https://[::ffff:127.0.0.3]/foo https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b] https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]]
    --- PASS: TestCheck/raw_config_checker/rejects_unique_local_ipv6_address (0.00s)
    === RUN   TestCheck/raw_config_checker/allows_public_ipv4_address
        main_test.go:216: Check url: https://8.8.8.8
        main_test.go:224: url detected ip 8.8.8.8, port 443
    --- PASS: TestCheck/raw_config_checker/allows_public_ipv4_address (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_malformed_url
        main_test.go:216: Check url: :/invalid-url
        main_test.go:221: got expected error: parse ":/invalid-url": missing protocol scheme
    --- PASS: TestCheck/raw_config_checker/rejects_malformed_url (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_unresolvable_host
        main_test.go:216: Check url: https://invalid-host-lookup.opensvc.com
        main_test.go:221: got expected error: lookup invalid-host-lookup.opensvc.com: no such host
    --- PASS: TestCheck/raw_config_checker/rejects_unresolvable_host (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_https_url_with_empty_host
        main_test.go:216: Check url: https://
        main_test.go:221: got expected error: empty host
    --- PASS: TestCheck/raw_config_checker/rejects_https_url_with_empty_host (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_empty_url
        main_test.go:216: Check url:
        main_test.go:221: got expected error: parse "": empty url
    --- PASS: TestCheck/raw_config_checker/rejects_empty_url (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_relative_path
        main_test.go:216: Check url: foo/bar
        main_test.go:221: got expected error: parse "foo/bar": invalid URI for request
    --- PASS: TestCheck/raw_config_checker/rejects_relative_path (0.00s)
    === RUN   TestCheck/raw_config_checker/rejects_dot_relative_path
        main_test.go:216: Check url: ./bar/foo
        main_test.go:221: got expected error: parse "./bar/foo": invalid URI for request
    --- PASS: TestCheck/raw_config_checker/rejects_dot_relative_path (0.00s)
    --- PASS: TestCheck/raw_config_checker (0.03s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allows_https_github_relay_v3_config_path
        main_test.go:236: Check url: https://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf: url https://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allows_https_github_relay_v3_config_path (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allows_https_github_host_only
        main_test.go:236: Check url: https://www.github.com
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://www.github.com: url https://www.github.com/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allows_https_github_host_only (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allows_https_github_host_with_trailing_slash
        main_test.go:236: Check url: https://www.github.com/
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://www.github.com/: url https://www.github.com/ is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allows_https_github_host_with_trailing_slash (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_http_scheme
        main_test.go:236: Check url: http://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url http://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf: url http://raw.githubusercontent.com/opensvc/opensvc_templates/refs/heads/main/relay-v3/relay-v3.conf is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_http_scheme (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_file_scheme_with_empty_host
        main_test.go:236: Check url: file:///tmp/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: empty host
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_file_scheme_with_empty_host (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_disallowed_port_on_allowlisted_host_path
        main_test.go:236: Check url: https://github.com:666/opensvc/om3
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:666/opensvc/om3: url https://github.com:666/opensvc/om3 is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_disallowed_port_on_allowlisted_host_path (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allows_exception_url_with_port_and_path
        main_test.go:236: Check url: https://github.com:8888/opensvc/om3
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:8888/opensvc/om3: url https://github.com:8888/opensvc/om3 is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allows_exception_url_with_port_and_path (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allows_subpath_under_allowlisted_path
        main_test.go:236: Check url: https://github.com:8888/opensvc/om3/bar
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:8888/opensvc/om3/bar: url https://github.com:8888/opensvc/om3/bar is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allows_subpath_under_allowlisted_path (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allows_normalized_path_within_allowlisted_path
        main_test.go:236: Check url: https://github.com:8888/opensvc/om3/../om3/bar
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:8888/opensvc/om3/../om3/bar: url https://github.com:8888/opensvc/om3/bar is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allows_normalized_path_within_allowlisted_path (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_normalized_path_outside_allowlisted_path
        main_test.go:236: Check url: https://github.com:8888/opensvc/om3/../oc3/bar
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:8888/opensvc/om3/../oc3/bar: url https://github.com:8888/opensvc/oc3/bar is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_normalized_path_outside_allowlisted_path (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_escaped_dotdot_path_escape
        main_test.go:236: Check url: https://github.com:8888/opensvc/om3/%2e%2e/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:8888/opensvc/om3/../foo: url https://github.com:8888/opensvc/foo is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_escaped_dotdot_path_escape (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_path_prefix_without_separator
        main_test.go:236: Check url: https://github.com:8888/opensvc/om3foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:8888/opensvc/om3foo: url https://github.com:8888/opensvc/om3foo is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_path_prefix_without_separator (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_path_outside_allowlisted_prefix
        main_test.go:236: Check url: https://github.com:8888/opensvc/foo/bar
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://github.com:8888/opensvc/foo/bar: url https://github.com:8888/opensvc/foo/bar is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_path_outside_allowlisted_prefix (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_file_scheme_with_remote_style_host
        main_test.go:236: Check url: file://raw.githubusercontent.com/tmp/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url file://raw.githubusercontent.com/tmp/foo: url file://raw.githubusercontent.com/tmp/foo is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_file_scheme_with_remote_style_host (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_unknown_scheme
        main_test.go:236: Check url: foo://raw.githubusercontent.com
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url foo://raw.githubusercontent.com: url foo://raw.githubusercontent.com/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_unknown_scheme (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_disallowed_port_on_allowlisted_raw_github_host
        main_test.go:236: Check url: https://raw.githubusercontent.com:8888
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://raw.githubusercontent.com:8888: url https://raw.githubusercontent.com:8888/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_disallowed_port_on_allowlisted_raw_github_host (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_non_allowlisted_host
        main_test.go:236: Check url: https://google.com
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://google.com: url https://google.com/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_non_allowlisted_host (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_ipv4_loopback_resolved_from_localhost
        main_test.go:236: Check url: https://localhost
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://localhost: url https://localhost/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_ipv4_loopback_resolved_from_localhost (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/block_cidr
        main_test.go:236: Check url: https://127.0.0.3/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://127.0.0.3/foo: url https://127.0.0.3/foo is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/block_cidr (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/block_cidr_v4_mapped_v6
        main_test.go:236: Check url: https://[::ffff:127.0.0.3]/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://[::ffff:127.0.0.3]/foo: url https://[::ffff:127.0.0.3]/foo is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/block_cidr_v4_mapped_v6 (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allow_cidr_exception_v4_mapped_v6
        main_test.go:236: Check url: https://[::ffff:127.0.0.2]/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://[::ffff:127.0.0.2]/foo: url https://[::ffff:127.0.0.2]/foo is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allow_cidr_exception_v4_mapped_v6 (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allow_cidr_exception
        main_test.go:236: Check url: https://127.0.0.2/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://127.0.0.2/foo: url https://127.0.0.2/foo is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allow_cidr_exception (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_ipv6_loopback
        main_test.go:236: Check url: https://[::1]
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://[::1]: url https://[::1]/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_ipv6_loopback (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/reject_ula_ipv6_cidr
        main_test.go:236: Check url: https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b]
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b]: url https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:626b]/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/reject_ula_ipv6_cidr (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allow_ula_ipv6_cidr_exception
        main_test.go:236: Check url: https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]: url https://[fd7a:115c:a1e0:ab12:4843:cd96:626b:430b]/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allow_ula_ipv6_cidr_exception (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_private_ipv4_address
        main_test.go:236: Check url: https://10.0.0.1
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://10.0.0.1: url https://10.0.0.1/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_private_ipv4_address (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_unique_local_ipv6_address
        main_test.go:236: Check url: https://[fc00::1]
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://[fc00::1]: url https://[fc00::1]/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_unique_local_ipv6_address (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/allows_public_ipv4_address
        main_test.go:236: Check url: https://8.8.8.8
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: reject url https://8.8.8.8: url https://8.8.8.8/. is blocked by pattern * and not in allowed list []
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/allows_public_ipv4_address (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_malformed_url
        main_test.go:236: Check url: :/invalid-url
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: parse ":/invalid-url": missing protocol scheme
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_malformed_url (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_unresolvable_host
        main_test.go:236: Check url: https://invalid-host-lookup.opensvc.com
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: lookup invalid-host-lookup.opensvc.com: no such host
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_unresolvable_host (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_https_url_with_empty_host
        main_test.go:236: Check url: https://
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: empty host
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_https_url_with_empty_host (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_empty_url
        main_test.go:236: Check url:
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: parse "": empty url
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_empty_url (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_relative_path
        main_test.go:236: Check url: foo/bar
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: parse "foo/bar": invalid URI for request
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_relative_path (0.00s)
    === RUN   TestCheck/zero_checker_must_reject_all_urls/rejects_dot_relative_path
        main_test.go:236: Check url: ./bar/foo
        main_test.go:239: url detected ip <nil>, port
        main_test.go:241: got expected error: parse "./bar/foo": invalid URI for request
    --- PASS: TestCheck/zero_checker_must_reject_all_urls/rejects_dot_relative_path (0.00s)
    --- PASS: TestCheck/zero_checker_must_reject_all_urls (0.02s)
    --- PASS: TestCheck (0.05s)
    PASS
@cgalibern
[util/uri] Introduce per-IP HTTP client for SSRF protections
e395c07 
This commit prevents TOCTOU bugs: GET request now use the returned
checker ip and it disable the allowed redirects by default.

Enhance `Fetch` to use a specific HTTP client for the resolved IP and
port from `Check`.

Add support for redirect control and network-layer
policy enforcement within `clientForIP`.
@cgalibern
[core/rawconfig] Document default SSRF policy configurations in `setS…
8d5292f 
…SRF`

// OSVC_SSRF_ALLOWED_URL = https://raw.githubusercontent.com/opensvc/opensvc_templates
// OSVC_SSRF_BLOCKED_URL = *
// OSVC_SSRF_BLOCKED_CIDR = 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 fc00::/7 127.0.0.0/8 ::1/128
// OSVC_SSRF_ALLOWED_CIDR =
// OSVC_SSRF_ENABLE_REDIRECTS = false
@cgalibern
[httppolicy] Improves error messages
f9794f9
@cgalibern
[util/httppolicy] Add URL normalization and enhanced pattern matching
b5a11c8 
Introduce `normalizePattern` and `normalizeURL` functions to standardize URL formats before matching.
Replace exact URL matches with flexible pattern-based matching using `fnmatch`. Update test cases to validate the new behavior.
@cgalibern
[core/rawconfig] Refactor setSSRF to simplify configuration handling
d5cfe6b 
This allows empty value definitions inside /etc/default/opensvc:
example:
     OSVC_SSRF_BLOCKED_URL=
     OSVC_SSRF_BLOCKED_CIDR=

Update `setSSRF` to take an environment map for configuration,
replacing individual parameter assignments with `getSSRFValue` utility.
Simplifies and consolidates SSRF policy initialization logic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@cgalibern