+ {"These hosts recently failed the TLS handshake while being intercepted, which often means they use certificate pinning. Excluding a host makes Privaxy tunnel it without inspection; ignoring it only hides it from this list."}
+
+ }
+ }
+}
From cd38612449180539fb346d1e52f3ea5f34ba2b30 Mon Sep 17 00:00:00 2001
From: Josh Mcdaniel <80354972+joshrmcdaniel@users.noreply.github.com>
Date: Wed, 29 Jul 2026 09:27:20 -0500
Subject: [PATCH 2/7] userscripting (#45)
* userscripting
* css in shadowdom
* perf
* mips
---
CHANGELOG.md | 207 +++
Cargo.lock | 11 +-
privaxy/Cargo.toml | 11 +-
privaxy/build.rs | 130 +-
privaxy/src/resources/procedural_cosmetics.js | 415 ++++-
privaxy/src/resources/userscript_shim.js | 888 ++++++++++
privaxy/src/server/blocker.rs | 501 +++---
privaxy/src/server/configuration/filter.rs | 2 +-
privaxy/src/server/configuration/mod.rs | 152 ++
privaxy/src/server/configuration/updater.rs | 36 +-
.../src/server/configuration/userscript.rs | 1520 +++++++++++++++++
privaxy/src/server/lib.rs | 179 +-
privaxy/src/server/proxy/gm/endpoint.rs | 447 +++++
privaxy/src/server/proxy/gm/fetch.rs | 486 ++++++
privaxy/src/server/proxy/gm/mod.rs | 24 +
privaxy/src/server/proxy/gm/storage.rs | 363 ++++
privaxy/src/server/proxy/gm/token.rs | 123 ++
privaxy/src/server/proxy/html_rewriter.rs | 557 ++++--
privaxy/src/server/proxy/mitm.rs | 4 +
privaxy/src/server/proxy/mod.rs | 2 +
privaxy/src/server/proxy/serve.rs | 143 +-
privaxy/src/server/proxy/userscripts.rs | 247 +++
privaxy/src/server/statistics.rs | 34 +-
privaxy/src/server/web_gui/mod.rs | 24 +
privaxy/src/server/web_gui/userscripts.rs | 594 +++++++
web_frontend/src/main.rs | 2 +
web_frontend/src/settings.rs | 9 +
web_frontend/src/userscript_edit.rs | 280 +++
web_frontend/src/userscripts.rs | 757 ++++++++
29 files changed, 7666 insertions(+), 482 deletions(-)
create mode 100644 privaxy/src/resources/userscript_shim.js
create mode 100644 privaxy/src/server/configuration/userscript.rs
create mode 100644 privaxy/src/server/proxy/gm/endpoint.rs
create mode 100644 privaxy/src/server/proxy/gm/fetch.rs
create mode 100644 privaxy/src/server/proxy/gm/mod.rs
create mode 100644 privaxy/src/server/proxy/gm/storage.rs
create mode 100644 privaxy/src/server/proxy/gm/token.rs
create mode 100644 privaxy/src/server/proxy/userscripts.rs
create mode 100644 privaxy/src/server/web_gui/userscripts.rs
create mode 100644 web_frontend/src/userscript_edit.rs
create mode 100644 web_frontend/src/userscripts.rs
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 91b69069..25493668 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,213 @@
## Unreleased
+- Userscript engine: Greasemonkey/Tampermonkey-style scripts injected into
+ matching pages, managed at runtime from the web UI.
+ - New **Settings → Userscripts** page: install a script by pasting it or by
+ URL (e.g. from Greasyfork), toggle scripts individually, edit a script's
+ source in place, and uninstall it. A master switch disables the whole
+ engine without clearing the per-script selection. Each entry shows its
+ version, `@run-at`, match patterns, grant count and `@noframes` state, and
+ a script whose stored body no longer parses is called out as not being
+ injected instead of failing silently.
+ - New `[userscripts]` configuration section holding the master `enabled` flag
+ and one `[[userscripts.scripts]]` entry per script. As with filter lists,
+ only metadata is stored in the configuration file; bodies live under
+ `userscripts/` in the configuration directory (override with
+ `PRIVAXY_USERSCRIPT_PATH`), keyed by a hash of the source URL so
+ re-installing the same script reuses its file. Configurations written
+ before this release parse unchanged and default to an enabled engine with
+ no scripts.
+ - Scripts are matched on the same canonical URL the adblock engine uses,
+ supporting `@match` (Chrome match-pattern syntax including `` and
+ `*.host` wildcards), `@include`/`@exclude` (globs, or regular expressions
+ written `/…/`), `@exclude-match`, `@run-at`
+ (`document-start`/`-body`/`-end`/`-idle`) and `@noframes`. Exclusions take
+ precedence over inclusions. A script declaring neither `@match` nor
+ `@include` — or with a malformed pattern or no metadata block — is rejected
+ with a `422` at install time rather than being stored and never firing.
+ - The in-page runtime provides `GM_info`, `unsafeWindow`, `GM_addStyle`
+ (nonce-stamped so it survives the page's CSP), `GM_log`, `GM_openInTab`,
+ `GM_setClipboard`, `GM_notification`, `GM_registerMenuCommand` and the
+ promise-based `GM.*` namespace.
+ - `@require` libraries and `@resource` payloads are fetched server-side and
+ cached on disk under `userscripts/assets/`, keyed by URL hash. Requires are
+ evaluated inside the script's own wrapper ahead of its body, so their
+ top-level declarations are visible to the script without leaking into the
+ page, and `GM_getResourceText`/`GM_getResourceURL` read the fetched
+ resources. Cached assets are never re-fetched (the convention is to pin a
+ versioned URL); delete the `assets` directory to refresh. An asset that
+ cannot be fetched degrades its script rather than dropping it, and the
+ failure is reported on the Userscripts page.
+ - `GM_setValue`/`GM_getValue`/`GM_deleteValue`/`GM_listValues` are persisted,
+ scoped per script, in `userscripts/gm_storage.json` (writes are coalesced
+ on a short debounce and written atomically; the configuration file is
+ deliberately not used, since these are written far too often to
+ re-serialize it each time). Because `GM_getValue` is synchronous in the GM
+ API, values are preloaded into each script's descriptor at injection time
+ and read from that snapshot in-page, so an ordinary `GM_getValue` costs no
+ request. Uninstalling a script drops its values.
+ - `GM_xmlhttpRequest` is relayed server-side, so it has no CORS restrictions
+ — the one capability a real content script cannot have. Three independent
+ controls gate it: the origin-bound token, the requesting script's own
+ `@connect` declarations (as Tampermonkey requires, so compatibility is
+ unaffected), and a filter rejecting loopback, RFC1918, carrier-grade NAT,
+ link-local (including the cloud metadata address) and IPv4-mapped
+ equivalents. Redirects are followed manually so `@connect` and address
+ filtering re-run on every hop rather than letting an allow-listed host
+ bounce the request to `127.0.0.1`. Methods are restricted, `Host` and
+ hop-by-hop request headers cannot be set, and responses are size-capped.
+ - `@resource` payloads may be binary. They are stored as bytes with the
+ content type they were served as; text small enough to matter is inlined
+ into the script's descriptor so `GM_getResourceText` stays synchronous,
+ while anything binary or oversized is reachable through
+ `GM_getResourceURL`, served from the reserved path with its original bytes,
+ content type and `X-Content-Type-Options: nosniff`. `GM_getResourceText` on
+ a binary resource returns `null` and logs a note pointing at
+ `GM_getResourceURL` rather than returning mojibake.
+ - `GM_addValueChangeListener`/`GM_removeValueChangeListener` are implemented.
+ Changes fire locally, reach other same-origin tabs over `BroadcastChannel`
+ (free, no request), and reach other origins and other devices behind the
+ same proxy by polling a read endpoint — which only runs while at least one
+ listener is registered, so a page with no listeners issues no extra
+ requests. The read endpoint requires the requesting URL to satisfy the
+ script's own `@match`/`@include`, so it can never reveal more than the
+ page's own descriptor already contained.
+ - `GM_registerMenuCommand` now has a real surface: a small floating menu,
+ injected only once a script actually registers a command, hosted in a closed
+ shadow root so neither the page's CSS nor Privaxy's can reach across.
+ Commands are still reachable from the console via
+ `__privaxyUserscriptMenu()`.
+ - Userscripts can be disabled for a single tab from that menu, backed by
+ `sessionStorage` — per-tab by construction, so the proxy needs no notion of
+ a tab. Other tabs are unaffected; `__privaxyUserscriptsEnableTab()` restores
+ them, since disabling removes the menu that turned them off.
+ - Userscripts installed from a URL are re-fetched on the same 24h timer as the
+ filter lists and recompiled in place, so upstream changes are picked up
+ without a restart. The refresh re-reads the configuration from disk rather
+ than reusing the updater's own copy: userscript changes deliberately bypass
+ that channel, so its copy is stale with respect to them and recompiling from
+ it would drop every script installed since startup.
+ - `@updateURL` and `@downloadURL` are honored, both defaulting to the URL the
+ script was installed from. When they differ, only the (small) `@updateURL`
+ document is fetched to compare `@version`, and the body is downloaded solely
+ when that version is newer — versions are ordered as dotted numbers, so
+ `1.2.10` correctly supersedes `1.2.9`, falling back to plain inequality for
+ schemes that cannot be ordered. Scripts that split metadata and body, a
+ common Greasyfork layout, previously re-downloaded the whole body on every
+ cycle.
+ - A **Check for updates** button on the Userscripts page refreshes on demand
+ instead of waiting out the timer, reporting per script whether it was
+ updated, already current, or failed and why. Unlike the periodic refresh it
+ holds the save lock, so a changed `@name` or `@version` is persisted.
+ - New `userscripts.allow_private_network_requests` setting (default off, with
+ a toggle on the Userscripts page) permits the relay to reach private
+ addresses. It is off by default because the relay runs server-side: the
+ proxy usually sits *inside* a LAN and can reach routers, admin panels and
+ metadata endpoints no page could contact. Changes apply immediately, with
+ no reload.
+ - Changes apply to the next page load with no reload or restart: compiled
+ scripts live in a shared store that each API mutation replaces in place,
+ and the store is also refreshed on `SIGHUP` so a hand-edited
+ `[userscripts]` section takes effect. Each script is emitted in its own
+ `nonce`d script element, so a syntax error in one script cannot abandon the
+ others or the ad-blocking payload, and the CSP nonce is kept in a closure
+ rather than published on `window`.
+ - Note that a userscript installed here runs on **every client behind the
+ proxy**, in the page's main world (a proxy has no isolated world to offer),
+ which is a wider blast radius than a browser extension installed in one
+ profile. The Userscripts page says so where scripts are added.
+ - **Not supported.** A userscript engine built into a proxy cannot reach full
+ Tampermonkey parity, and some of what is missing is structural rather than
+ unfinished. Known gaps, so a script that misbehaves can be diagnosed instead
+ of guessed at:
+ - *No isolated world, and there cannot be one.* Scripts run in the page's
+ main world, so `unsafeWindow === window`, page scripts can read and clobber
+ anything a userscript leaves reachable, and anti-adblock can detect the
+ injection. Each script is still wrapped in its own function, so its `var`,
+ `let`, `const`, `function` and `class` declarations do not leak to the page
+ — only an undeclared assignment or an explicit `window.x = …` does.
+ - *Pages the proxy never sees get nothing.* A site with an active service
+ worker serving navigations from cache, `file://`, `chrome://`, browser-cache
+ hits and any traffic not routed through Privaxy are all invisible to it, so
+ no script runs there. A browser extension sees all of them.
+ - *`@grant` is parsed and displayed but not enforced.* Every script receives
+ every implemented API regardless of what it declared, including
+ `@grant none`. This is deliberately forgiving — a script that forgot to
+ declare a grant still works — but it is a deviation.
+ - *Not implemented:* `GM_cookie` (the proxy has no cookie jar for the
+ browser's cookies; it only sees `Cookie` headers in flight),
+ `GM_getTab`/`GM_saveTab`/`GM_getTabs` (no tab identity exists on the proxy
+ side), `GM_download` (it would mean the proxy writing files to its own disk
+ on a page's behalf), `GM_addElement`, the batch `GM_setValues`/`GM_getValues`
+ forms, and `window.onurlchange`.
+ - *`GM_notification` writes to the console* rather than raising a real
+ notification, and `GM_setClipboard` needs a user gesture like any page-context
+ clipboard write.
+ - *Metadata ignored:* `@sandbox`, `@unwrap`, `@top-level-await`, `@icon`,
+ `@supportURL`, `@antifeature`. Unknown directives are skipped, not rejected.
+ - *`@resource` payloads are byte-exact but not inlined as text when binary or
+ over 256 KB* — `GM_getResourceText` returns `null` for those and the data is
+ reachable only through `GM_getResourceURL`.
+ - *`GM_xmlhttpRequest` is not a transparent `XMLHttpRequest`.* Responses are
+ decoded as lossy UTF-8, so binary bodies are unusable; `abort()` only
+ suppresses the callbacks, since the server-side request is already in
+ flight; there is no `onprogress`/`onreadystatechange`; redirects are capped
+ at 5 hops, timeouts at 60s (default 30s) and responses at 8 MB; and the
+ request only reaches hosts the script declared with `@connect`.
+ - *Value-change notification is not instant across origins.* Same-origin tabs
+ are updated immediately over `BroadcastChannel`; a change made on a
+ different origin or another device is picked up by a 15s poll, and only
+ while a listener is registered.
+ - *Per-tab disable is per-tab **per origin*** — it is `sessionStorage`, so
+ disabling on one site does not disable on another in the same tab.
+ - *Storage limits:* 1000 keys per script and 64 KB per value; a script body
+ or fetched `@require`/`@resource` is capped at 2 MB. Writes are flushed on a
+ 500 ms debounce, so values set immediately before a crash can be lost.
+ - *No script ordering or import/export in the UI.* Injection order is
+ configuration order, changeable only by editing the file.
+- Proxy performance overhaul:
+ - *HTML responses now stream.* The proxy previously withheld an HTML
+ response — status line, headers and all — until the entire upstream
+ document had been downloaded and fed through the rewriter, so the browser
+ could not start parsing (or prefetching subresources) until the last
+ upstream byte arrived. The rewritten document now streams to the client as
+ it is produced, and the rewriter pipeline is bounded end-to-end, so a slow
+ client backpressures the upstream download instead of the whole document
+ buffering in memory.
+ - *WebSocket tunnels no longer squeeze through a 32-byte buffer.* The duplex
+ buffer bridging the client and upstream halves of an upgraded connection
+ was 32 bytes, forcing a task wakeup roughly every 32 bytes transferred;
+ it is now 64 KiB.
+ - *The adblock engine is shared, not funneled through one thread.* The
+ `single-thread` adblock feature is dropped; the engine (Send + Sync) is
+ now called directly from request tasks, removing a channel round-trip and
+ two cross-thread handoffs from every request. Matching itself still
+ serializes briefly on the engine's internal regex-manager lock — the same
+ one-core ceiling as the old blocker thread, far above proxy request rates
+ — but the per-request overhead around it is gone. Filter-list updates
+ build the replacement engine on the blocking pool and swap it in
+ atomically, so requests keep matching against the old engine during a
+ multi-second list rebuild instead of stalling behind it.
+ - *One cosmetic lookup per page instead of two.* The URL-scoped cosmetic
+ lookup (`url_cosmetic_resources`) ran once for the `` injection and
+ again at end-of-body; the end-of-body pass now reuses the first lookup and
+ only resolves the generic class/id-indexed selectors on top.
+ - Dashboard events are only constructed when a client is actually watching
+ the live requests feed; statistics counters are atomics instead of
+ mutexes; the HTML rewriter no longer compiles a (redundant) regex per
+ response and scans each element once instead of twice.
+ - New 5-minute read timeout on proxied requests bounds a peer that stops
+ sending mid-response without closing (previously such a request hung
+ forever); generous enough not to disturb long-polls or quiet SSE streams.
+- A configuration file that fails to parse no longer takes the server down.
+ `read_configuration` unwrapped the parse error, so a hand-edited file with (for
+ example) a duplicate TOML key panicked a worker on `SIGHUP` and killed both the
+ proxy and web-UI loops while the process kept running — both ports stopped
+ listening with no way back except a restart. The last configuration that parsed
+ is now kept and reused, so a reload over a broken file logs the error and
+ carries on serving with the previous settings, then picks up the corrected file
+ on the next reload. The CA reload in the same path no longer unwraps either.
- The PAC route now also answers at `/wpad.dat`, so DNS-based WPAD
auto-discovery (`http://wpad./wpad.dat`) can point straight
at Privaxy without needing a rewrite in a fronting reverse proxy.
diff --git a/Cargo.lock b/Cargo.lock
index 0c26f8ec..580489f5 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -418,15 +418,6 @@ dependencies = [
"cfg-if",
]
-[[package]]
-name = "crossbeam-channel"
-version = "0.5.15"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2"
-dependencies = [
- "crossbeam-utils",
-]
-
[[package]]
name = "crossbeam-deque"
version = "0.8.6"
@@ -2179,7 +2170,6 @@ dependencies = [
"base64",
"bytes",
"chrono",
- "crossbeam-channel",
"dirs",
"env_logger",
"filterlists-api",
@@ -2201,6 +2191,7 @@ dependencies = [
"mime_guess",
"once_cell",
"openssl",
+ "portable-atomic",
"regex",
"reqwest",
"rustls",
diff --git a/privaxy/Cargo.toml b/privaxy/Cargo.toml
index dc2a7a56..94c35d52 100644
--- a/privaxy/Cargo.toml
+++ b/privaxy/Cargo.toml
@@ -70,7 +70,14 @@ tokio-util = { version = "0.7.4", features = ["full"] }
tokio-stream = "0.1"
bytes = "1"
socket2 = "0.5"
-adblock = { version = "0.12.5" }
+# default-features = false only to drop `single-thread`: the multi-thread
+# engine is Send + Sync, which lets requests share one engine directly instead
+# of funneling through a dedicated blocker thread. The other two features are
+# adblock defaults, restated because default-features is all-or-nothing.
+adblock = { version = "0.12.5", default-features = false, features = [
+ "embedded-domain-resolver",
+ "full-regex-handling",
+] }
openssl = { version = "0.10.43", features = ["vendored"] }
include_dir = "0.7.3"
chrono = { version = "0.4.23", features = ["serde"] }
@@ -100,10 +107,10 @@ hyper-rustls = { version = "0.27", default-features = false, features = [
log = "0.4.17"
env_logger = "0.11.10"
uluru = "3.1.0"
+portable-atomic = "1"
regex = "1.7.0"
lazy_static = "1.4.0"
lol_html = "1.2.1"
-crossbeam-channel = "0.5.6"
thiserror = "2"
url = "2.3.1"
futures = "0.3.25"
diff --git a/privaxy/build.rs b/privaxy/build.rs
index 99bfefaf..3043b920 100644
--- a/privaxy/build.rs
+++ b/privaxy/build.rs
@@ -1,12 +1,136 @@
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
use std::process::Command;
fn main() {
let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+ let out_dir = PathBuf::from(std::env::var("OUT_DIR").expect("cargo provides OUT_DIR"));
+
+ build_procedural_shim(&manifest_dir, &out_dir);
+ build_scriptlet_resources(&manifest_dir, &out_dir);
+}
+
+/// Strip the comments out of the procedural-cosmetics shim before it is embedded.
+///
+/// The shim is injected inline into every HTML response that has cosmetic rules,
+/// and the proxy re-sends responses uncompressed, so its comments would be real
+/// bytes on the wire on every page load — roughly two thirds of the file. Keeping
+/// them in the source and dropping them here means the reasoning lives next to
+/// the code without being paid for at runtime.
+///
+/// Deliberately not done with Node: the cross-compile path in
+/// `build_scriptlet_resources` exists precisely because Node isn't always
+/// available, and the shim has to be embedded in every build.
+fn build_procedural_shim(manifest_dir: &Path, out_dir: &Path) {
+ let source_path = manifest_dir.join("src/resources/procedural_cosmetics.js");
+ println!("cargo:rerun-if-changed={}", source_path.display());
+
+ let source = std::fs::read_to_string(&source_path).unwrap_or_else(|e| {
+ panic!(
+ "failed to read the procedural shim at {}: {e}",
+ source_path.display()
+ )
+ });
+
+ let embedded = match strip_whole_line_comments(&source) {
+ Some(stripped) => stripped,
+ None => {
+ // Embedding the file verbatim is always correct, only larger, so an
+ // unrecognised construct degrades instead of risking a corrupted shim
+ // on every page.
+ println!(
+ "cargo:warning=procedural_cosmetics.js contains a construct the comment \
+ stripper does not handle (a multi-line string, or a comment sharing a line \
+ with code); embedding it verbatim. See strip_whole_line_comments in \
+ privaxy/build.rs."
+ );
+ source
+ }
+ };
+
+ std::fs::write(out_dir.join("procedural_cosmetics.js"), embedded)
+ .expect("failed to write the stripped procedural shim to OUT_DIR");
+}
+
+/// Remove comment-only lines, blank lines, and indentation from JavaScript.
+///
+/// This is a line-oriented pass, not a JavaScript lexer: it decides what to drop
+/// purely from how a line *starts*, and never inspects the interior of a line it
+/// keeps. That is what makes it safe without parsing — a `//` inside a string or
+/// a regex literal cannot be mistaken for a comment, because such a line is kept
+/// whole.
+///
+/// It returns `None` rather than guessing whenever it meets something that would
+/// invalidate that reasoning:
+///
+/// * a backtick, which may open a template literal spanning lines: a line
+/// inside one could begin with `//` and would then be dropped wrongly;
+/// * a trailing backslash, i.e. a string continued onto the next line, for the
+/// same reason;
+/// * a comment sharing a line with code, where telling comment from string
+/// genuinely does need a lexer.
+///
+/// Line numbers shift, which costs nothing here: the shim is concatenated after
+/// the scriptlet payload before injection, so they never matched the file anyway.
+fn strip_whole_line_comments(source: &str) -> Option {
+ let mut out = String::with_capacity(source.len());
+ let mut in_block_comment = false;
+
+ for line in source.lines() {
+ let trimmed = line.trim();
+
+ if in_block_comment {
+ match trimmed.find("*/") {
+ // Code trailing the end of a block comment needs a lexer to split.
+ Some(end) if end + 2 != trimmed.len() => return None,
+ Some(_) => in_block_comment = false,
+ None => {}
+ }
+ continue;
+ }
+
+ if trimmed.starts_with("//") {
+ continue;
+ }
+
+ if trimmed.starts_with("/*") {
+ match trimmed.find("*/") {
+ Some(end) if end + 2 != trimmed.len() => return None,
+ // `/*/` both starts and ends with a delimiter without being a
+ // complete comment, hence the length check.
+ Some(_) if trimmed.len() >= 4 => {}
+ _ => in_block_comment = true,
+ }
+ continue;
+ }
+
+ if trimmed.is_empty() {
+ continue;
+ }
+
+ if trimmed.contains('`')
+ || trimmed.ends_with('\\')
+ || trimmed.contains("//")
+ || trimmed.contains("/*")
+ {
+ return None;
+ }
+
+ out.push_str(trimmed);
+ out.push('\n');
+ }
+
+ // An unterminated block comment means the file was not what we assumed.
+ if in_block_comment {
+ return None;
+ }
+
+ Some(out)
+}
+
+fn build_scriptlet_resources(manifest_dir: &Path, out_dir: &Path) {
let scriptlets_src = manifest_dir.join("src/resources/vendor/ublock/scriptlets.js");
let builder = manifest_dir.join("build-scriptlets.mjs");
- let out_dir = std::env::var("OUT_DIR").expect("cargo provides OUT_DIR");
- let out_path = PathBuf::from(&out_dir).join("scriptlets-resources.json");
+ let out_path = out_dir.join("scriptlets-resources.json");
println!("cargo:rerun-if-changed={}", scriptlets_src.display());
println!("cargo:rerun-if-changed={}", builder.display());
diff --git a/privaxy/src/resources/procedural_cosmetics.js b/privaxy/src/resources/procedural_cosmetics.js
index 5be040f7..4139546d 100644
--- a/privaxy/src/resources/procedural_cosmetics.js
+++ b/privaxy/src/resources/procedural_cosmetics.js
@@ -7,6 +7,20 @@
* This shim receives those rules as JSON and applies them on load and on every
* subsequent DOM mutation.
*
+ * It also adopts the plain-CSS cosmetic rules into every shadow root it finds.
+ * That server-side
"#
);
diff --git a/privaxy/src/server/proxy/mitm.rs b/privaxy/src/server/proxy/mitm.rs
index 8e532ed5..0b1b2b86 100644
--- a/privaxy/src/server/proxy/mitm.rs
+++ b/privaxy/src/server/proxy/mitm.rs
@@ -1,5 +1,6 @@
use super::serve::UpgradeClient;
use super::tls_failures::TlsFailureStore;
+use super::userscripts::UserScriptContext;
use super::{empty_body, exclusions::LocalExclusionStore, serve::serve, ProxyBody};
use crate::{
blocker::AdblockRequester, cert::CertCache, configuration::DohConfig, statistics::Statistics,
@@ -35,6 +36,7 @@ pub(crate) async fn serve_mitm_session(
scriptlet_debug_logging: bool,
tls_failure_store: TlsFailureStore,
gui_base_url: Option,
+ user_scripts: UserScriptContext,
) -> Result, hyper::Error> {
let raw_authority = match req.uri().authority().cloned() {
Some(authority) => authority,
@@ -125,6 +127,7 @@ pub(crate) async fn serve_mitm_session(
doh_config.clone(),
scriptlet_debug_logging,
gui_base_url.clone(),
+ user_scripts.clone(),
)
}),
)
@@ -208,6 +211,7 @@ pub(crate) async fn serve_mitm_session(
doh_config,
scriptlet_debug_logging,
gui_base_url,
+ user_scripts,
)
.await
}
diff --git a/privaxy/src/server/proxy/mod.rs b/privaxy/src/server/proxy/mod.rs
index 09a60221..0c162b07 100644
--- a/privaxy/src/server/proxy/mod.rs
+++ b/privaxy/src/server/proxy/mod.rs
@@ -3,8 +3,10 @@ pub(crate) mod mitm;
pub(crate) mod serve;
pub(crate) use mitm::serve_mitm_session;
pub(crate) mod exclusions;
+pub(crate) mod gm;
pub(crate) mod html_rewriter;
pub(crate) mod tls_failures;
+pub(crate) mod userscripts;
use bytes::Bytes;
use http_body_util::{combinators::BoxBody, BodyExt, Empty, Full, StreamBody};
diff --git a/privaxy/src/server/proxy/serve.rs b/privaxy/src/server/proxy/serve.rs
index 42536c44..ded75085 100644
--- a/privaxy/src/server/proxy/serve.rs
+++ b/privaxy/src/server/proxy/serve.rs
@@ -1,5 +1,7 @@
use super::doh::{self, DohAction};
+use super::gm::endpoint as gm_endpoint;
use super::html_rewriter::Rewriter;
+use super::userscripts::UserScriptContext;
use super::{body_channel, boxed_incoming, empty_body, full_body, BodySender, ProxyBody};
use crate::blocker::AdblockRequester;
use crate::configuration::DohConfig;
@@ -11,7 +13,7 @@ use bytes::Bytes;
use futures::TryStreamExt;
use http::uri::{Authority, Scheme};
use http::{HeaderMap, HeaderValue, Request, Response, StatusCode, Uri};
-use http_body_util::BodyStream;
+use http_body_util::{BodyExt, BodyStream, Limited};
use hyper::body::{Frame, Incoming};
use hyper_rustls::HttpsConnector;
use hyper_util::client::legacy::connect::HttpConnector;
@@ -19,6 +21,7 @@ use hyper_util::client::legacy::Client as HyperClient;
use hyper_util::rt::TokioIo;
use std::net::IpAddr;
use tokio::sync::broadcast;
+use url::Url;
/// Type of the hyper client used for upgrade tunneling (websockets, etc.).
pub(crate) type UpgradeClient = HyperClient, ProxyBody>;
@@ -45,6 +48,10 @@ const CSP_HEADERS: [&str; 3] = [
"x-webkit-csp",
];
+/// Cap on a buffered request body for the reserved endpoints. Generous for a
+/// batch of `GM_setValue` writes and far below anything worth buffering.
+const MAX_RESERVED_ENDPOINT_BODY_BYTES: usize = 256 * 1024;
+
/// 16 random bytes → 22 chars of url-safe base64. Plenty of entropy and
/// avoids `=` padding that some CSP parsers historically choked on.
fn generate_csp_nonce() -> String {
@@ -277,6 +284,7 @@ pub(crate) async fn serve(
doh_config: DohConfig,
scriptlet_debug_logging: bool,
gui_base_url: Option,
+ user_scripts: UserScriptContext,
) -> Result, hyper::Error> {
let scheme_string = scheme.to_string();
@@ -299,6 +307,25 @@ pub(crate) async fn serve(
return Ok(perform_two_ends_upgrade(request, uri, hyper_client).await);
}
+ // Requests to the reserved path are answered by the proxy on the page's own
+ // origin and never forwarded upstream, which is what lets a userscript
+ // running in the page's main world reach Privaxy without CORS. Handled
+ // before any statistics counting: these are not proxied traffic.
+ if gm_endpoint::is_reserved(uri.path()) {
+ let (parts, body) = request.into_parts();
+ // Bounded so a hostile page cannot make the proxy buffer an unbounded
+ // body; an oversized request simply fails to parse below.
+ let collected = Limited::new(body, MAX_RESERVED_ENDPOINT_BODY_BYTES)
+ .collect()
+ .await
+ .map(|collected| collected.to_bytes())
+ .unwrap_or_default();
+
+ return Ok(
+ gm_endpoint::handle(&uri, &parts.method, &collected, &user_scripts, &client).await,
+ );
+ }
+
let (mut parts, body) = request.into_parts();
parts.uri = uri.clone();
@@ -310,7 +337,7 @@ pub(crate) async fn serve(
statistics.increment_top_clients(client_ip_address);
- let request_type = request_type_from_headers(req.headers()).to_string();
+ let request_type = request_type_from_headers(req.headers());
// Canonical URL (default port stripped) for all adblock-engine matching —
// see `url_for_matching`. The raw `uri` (which may carry `:443`) is still
@@ -322,36 +349,40 @@ pub(crate) async fn serve(
let doh_action = doh::classify(&doh_config, req.headers(), &uri);
if let DohAction::Block = &doh_action {
log::debug!("Refusing DoH request: {}", uri);
+ // Events only feed the live requests view; when nobody is subscribed,
+ // building one (two Strings and a timestamp) is wasted work.
+ if broadcast_sender.receiver_count() > 0 {
+ let _result = broadcast_sender.send(Event {
+ now: chrono::Utc::now(),
+ method: req.method().to_string(),
+ url: req.uri().to_string(),
+ is_request_blocked: true,
+ });
+ }
+ statistics.increment_blocked_requests();
+ return Ok(get_empty_response(StatusCode::BAD_GATEWAY));
+ }
+
+ let (is_request_blocked, blocker_result) = adblock_requester.is_network_url_blocked(
+ &match_url,
+ match req.headers().get(http::header::REFERER) {
+ Some(referer) => referer.to_str().unwrap_or(&match_url),
+ // When no referer, we default to `uri` as we otherwise may get many false
+ // positives due to the blocker thinking it's third party requests.
+ None => &match_url,
+ },
+ request_type,
+ );
+
+ if broadcast_sender.receiver_count() > 0 {
let _result = broadcast_sender.send(Event {
now: chrono::Utc::now(),
method: req.method().to_string(),
url: req.uri().to_string(),
- is_request_blocked: true,
+ is_request_blocked,
});
- statistics.increment_blocked_requests();
- return Ok(get_empty_response(StatusCode::BAD_GATEWAY));
}
- let (is_request_blocked, blocker_result) = adblock_requester
- .is_network_url_blocked(
- match_url.clone(),
- match req.headers().get(http::header::REFERER) {
- Some(referer) => referer.to_str().unwrap().to_string(),
- // When no referer, we default to `uri` as we otherwise may get many false
- // positives due to the blocker thinking it's third party requests.
- None => match_url.clone(),
- },
- request_type.clone(),
- )
- .await;
-
- let _result = broadcast_sender.send(Event {
- now: chrono::Utc::now(),
- method: req.method().to_string(),
- url: req.uri().to_string(),
- is_request_blocked,
- });
-
if is_request_blocked {
statistics.increment_blocked_requests();
statistics.increment_top_blocked_paths(format!(
@@ -477,36 +508,70 @@ pub(crate) async fn serve(
let new_response = Response::from_parts(parts, new_new_body);
if is_html {
- let (sender_rewriter, receiver_rewriter) = crossbeam_channel::unbounded::();
+ // Bounded so the whole backpressure chain holds: client ← response body
+ // ← rewriter ← this channel ← upstream. Before, every hop here was
+ // unbounded and the entire document buffered in memory.
+ let (sender_rewriter, receiver_rewriter) = tokio::sync::mpsc::channel::(32);
// Resolve the URL-scoped payloads up-front so the rewriter can prepend
// them inside before any page scripts execute: the uBO scriptlet
// and the procedural cosmetic filters (both URL-specific, not dependent
- // on collected IDs/classes). The end-of-body cosmetic lookup still runs
- // for hide/style selectors, which do depend on collected IDs/classes.
+ // on collected IDs/classes). This is the single `url_cosmetic_resources`
+ // lookup for the page — the end-of-body pass only resolves the generic
+ // class/id-indexed selectors on top of it, using the exception set
+ // carried in this result.
let head_cosmetics = adblock_requester
- .get_cosmetic_response(match_url.clone(), Vec::new(), Vec::new())
+ .get_cosmetic_response(match_url.clone())
.await;
+ // Userscripts are matched against the same canonical URL the adblock
+ // engine uses. The store is consulted per request rather than captured
+ // once per proxy start, so scripts added or toggled in the web UI apply
+ // to the very next page load without a reload.
+ let matched_user_scripts = if user_scripts.store.is_empty() {
+ Vec::new()
+ } else {
+ match Url::parse(&match_url) {
+ Ok(url) => user_scripts.store.matching(&url),
+ Err(err) => {
+ log::debug!("Not matching userscripts against {match_url}: {err}");
+ Vec::new()
+ }
+ }
+ };
+
+ // Minted per page and handed to the in-page runtime so it can persist
+ // GM values. Bound to this origin; see `userscript_token`.
+ let endpoint_token = gm_endpoint::origin_of(&uri)
+ .map(|origin| super::gm::token::mint(&origin, &user_scripts.endpoint_signing_key));
+
let rewriter = Rewriter::new(
- match_url.clone(),
adblock_requester,
receiver_rewriter,
sender,
statistics,
csp_nonce.expect("csp_nonce is Some whenever is_html"),
- head_cosmetics.injected_script,
- head_cosmetics.procedural_filters,
+ head_cosmetics,
scriptlet_debug_logging,
+ matched_user_scripts,
+ user_scripts.gm_storage.clone(),
+ endpoint_token,
);
tokio::task::spawn_blocking(|| rewriter.rewrite());
- while let Ok(Some(chunk)) = response.chunk().await {
- if let Err(_err) = sender_rewriter.send(chunk) {
- break;
+ // Drain the upstream body on its own task so the response (headers plus
+ // the streaming rewritten body) is returned to the client immediately.
+ // Holding the response until the whole document had been downloaded
+ // meant the browser could not start parsing — or prefetching
+ // subresources — until the very last upstream byte had arrived.
+ tokio::spawn(async move {
+ while let Ok(Some(chunk)) = response.chunk().await {
+ if sender_rewriter.send(chunk).await.is_err() {
+ break;
+ }
}
- }
+ });
return Ok(new_response);
}
@@ -645,7 +710,11 @@ async fn perform_two_ends_upgrade(
uri: Uri,
hyper_client: UpgradeClient,
) -> Response {
- let (mut duplex_client, mut duplex_server) = tokio::io::duplex(32);
+ // The duplex buffer caps how many bytes can be in flight between the two
+ // `copy_bidirectional` tasks bridging client and upstream. A tiny buffer
+ // forces the tunnel to ping-pong wakeups every few bytes, throttling
+ // WebSocket throughput; 64 KiB matches a typical socket buffer.
+ let (mut duplex_client, mut duplex_server) = tokio::io::duplex(64 * 1024);
// Captured for log context; `uri` is moved into `new_request` below.
let request_uri = uri.to_string();
diff --git a/privaxy/src/server/proxy/userscripts.rs b/privaxy/src/server/proxy/userscripts.rs
new file mode 100644
index 00000000..68f96d5c
--- /dev/null
+++ b/privaxy/src/server/proxy/userscripts.rs
@@ -0,0 +1,247 @@
+//! Runtime store of compiled userscripts.
+//!
+//! Userscript changes made in the web UI must take effect on the next request,
+//! not on the next reload. The proxy loop reads most configuration once per
+//! (re)start — `scriptlet_debug_logging` is read that way in `lib.rs` — so
+//! userscripts instead live behind this cheaply-cloneable handle, following the
+//! same pattern as [`crate::proxy::exclusions::LocalExclusionStore`] and
+//! [`crate::proxy::tls_failures::TlsFailureStore`]. Each API mutation replaces
+//! the contents in place and every subsequent page load sees the new set.
+
+use super::gm::storage::GmStorageStore;
+use crate::configuration::{CompiledUserScript, Configuration};
+use std::sync::atomic::{AtomicBool, Ordering};
+use std::sync::{Arc, RwLock};
+use url::Url;
+
+/// Everything the request path needs in order to serve userscripts.
+///
+/// Bundled rather than passed as three more arguments through
+/// `serve_mitm_session` -> `serve` -> `Rewriter`, which already carry more
+/// parameters than is comfortable.
+#[derive(Debug, Clone)]
+pub struct UserScriptContext {
+ /// Compiled scripts, replaced in place whenever the configuration changes.
+ pub store: UserScriptStore,
+ /// Persistent `GM_setValue` data.
+ pub gm_storage: GmStorageStore,
+ /// Key the reserved-endpoint tokens are derived from. Read once per proxy
+ /// (re)start, so both minting and verification always use the same value.
+ pub endpoint_signing_key: String,
+ /// Mirrors `userscripts.allow_private_network_requests`, consulted by the
+ /// `GM_xmlhttpRequest` relay.
+ ///
+ /// Shared and atomic rather than a captured `bool`: the API updates it in
+ /// place so toggling it in the web UI takes effect immediately, like every
+ /// other userscript setting, instead of waiting for a reload.
+ pub allow_private_network_requests: PrivateNetworkAccess,
+}
+
+/// Live switch for the relay's private-address filter.
+#[derive(Debug, Clone, Default)]
+pub struct PrivateNetworkAccess(Arc);
+
+impl PrivateNetworkAccess {
+ pub fn new(allowed: bool) -> Self {
+ Self(Arc::new(AtomicBool::new(allowed)))
+ }
+
+ pub fn is_allowed(&self) -> bool {
+ self.0.load(Ordering::Relaxed)
+ }
+
+ pub fn set(&self, allowed: bool) {
+ self.0.store(allowed, Ordering::Relaxed);
+ }
+}
+
+/// Shared, cheaply-cloneable set of compiled userscripts.
+///
+/// Scripts are held behind an `Arc` so matching a URL only clones pointers,
+/// never script bodies, and the read lock is released before any of them is
+/// used. Lock scopes are short and never held across an await point.
+#[derive(Debug, Clone, Default)]
+pub struct UserScriptStore(Arc>>>);
+
+impl UserScriptStore {
+ pub fn new(scripts: Vec) -> Self {
+ Self(Arc::new(RwLock::new(
+ scripts.into_iter().map(Arc::new).collect(),
+ )))
+ }
+
+ /// Swap in a freshly compiled set, discarding the previous one.
+ pub fn replace(&self, scripts: Vec) {
+ let mut guard = self.0.write().unwrap();
+ *guard = scripts.into_iter().map(Arc::new).collect();
+ }
+
+ /// Scripts whose `@match`/`@include` declarations select `url`, in
+ /// configuration order.
+ pub fn matching(&self, url: &Url) -> Vec> {
+ let guard = self.0.read().unwrap();
+
+ guard
+ .iter()
+ .filter(|script| script.matches(url))
+ .cloned()
+ .collect()
+ }
+
+ /// Whether any script is loaded. Lets the request path skip URL parsing
+ /// entirely in the common case of no userscripts installed.
+ pub fn is_empty(&self) -> bool {
+ self.0.read().unwrap().is_empty()
+ }
+
+ /// The compiled form of one script, by file name. Used by the API to report
+ /// compile warnings without redoing the work.
+ pub fn find(&self, file_name: &str) -> Option> {
+ self.0
+ .read()
+ .unwrap()
+ .iter()
+ .find(|script| script.file_name == file_name)
+ .cloned()
+ }
+}
+
+/// Compile every active script in `configuration`, reading bodies from disk and
+/// resolving each script's `@require`/`@resource` assets (cached on disk after
+/// the first fetch).
+///
+/// A script that fails to load or no longer parses is logged and dropped rather
+/// than aborting the whole rebuild — the same policy
+/// [`crate::configuration::get_filters_content`] applies to filter lists. An
+/// asset that fails to load only degrades its own script; see
+/// [`CompiledUserScript::resolve_assets`].
+pub async fn compile_active_userscripts(
+ configuration: &Configuration,
+ http_client: &reqwest::Client,
+) -> Vec {
+ let mut compiled = Vec::new();
+
+ for script in configuration.userscripts.active_scripts() {
+ let body = match script.read_body().await {
+ Ok(body) => body,
+ Err(err) => {
+ log::warn!(
+ "Dropping userscript '{}' whose body could not be read: {err}",
+ script.title
+ );
+ continue;
+ }
+ };
+
+ match CompiledUserScript::new(script, body) {
+ Ok(mut compiled_script) => {
+ compiled_script.resolve_assets(http_client).await;
+
+ for warning in &compiled_script.warnings {
+ log::warn!("Userscript '{}': {warning}", compiled_script.title);
+ }
+
+ compiled.push(compiled_script);
+ }
+ Err(err) => log::warn!(
+ "Dropping userscript '{}' that no longer parses: {err}",
+ script.title
+ ),
+ }
+ }
+
+ log::debug!("Compiled {} userscript(s)", compiled.len());
+
+ compiled
+}
+
+/// Rebuild the store from `configuration`. Called at startup and after every
+/// mutation from the API so changes apply without a proxy restart.
+pub async fn reload_userscripts(
+ store: &UserScriptStore,
+ configuration: &Configuration,
+ http_client: &reqwest::Client,
+) {
+ store.replace(compile_active_userscripts(configuration, http_client).await);
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::configuration::UserScript;
+
+ fn compiled(name: &str, match_pattern: &str) -> CompiledUserScript {
+ let body = format!(
+ "// ==UserScript==\n// @name {name}\n// @match {match_pattern}\n// ==/UserScript==\nvoid 0;\n"
+ );
+ let script = UserScript {
+ enabled: true,
+ title: name.to_string(),
+ file_name: format!("{name}.user.js"),
+ url: None,
+ };
+
+ CompiledUserScript::new(&script, body).expect("compiles")
+ }
+
+ #[test]
+ fn matching_selects_only_scripts_whose_patterns_match() {
+ let store = UserScriptStore::new(vec![
+ compiled("example-only", "https://example.com/*"),
+ compiled("everywhere", ""),
+ ]);
+
+ let matched = store.matching(&Url::parse("https://example.com/page").unwrap());
+ assert_eq!(matched.len(), 2);
+
+ let matched = store.matching(&Url::parse("https://other.test/page").unwrap());
+ assert_eq!(matched.len(), 1);
+ assert_eq!(matched[0].title, "everywhere");
+ }
+
+ /// A `@require` that cannot be fetched must degrade the script, not drop
+ /// it: the body still runs and the failure is reported as a warning.
+ #[tokio::test]
+ async fn unreachable_require_becomes_a_warning() {
+ let body = "// ==UserScript==\n// @name Needs a library\n// @match \n\
+ // @require http://127.0.0.1:1/never-listening.js\n// ==/UserScript==\nvoid 0;\n";
+ let script = UserScript {
+ enabled: true,
+ title: "Needs a library".to_string(),
+ file_name: "needs-a-library.user.js".to_string(),
+ url: None,
+ };
+
+ let mut compiled = CompiledUserScript::new(&script, body.to_string()).expect("compiles");
+ compiled.resolve_assets(&reqwest::Client::new()).await;
+
+ assert!(compiled.requires.is_empty());
+ assert_eq!(compiled.warnings.len(), 1);
+ assert!(
+ compiled.warnings[0].contains("@require"),
+ "warning should name the failing directive: {}",
+ compiled.warnings[0]
+ );
+ // Still injectable.
+ assert!(compiled.matches(&Url::parse("https://anything.test/").unwrap()));
+ }
+
+ /// The store is what makes web-UI changes take effect without a reload, so
+ /// a replacement must be visible through an already-cloned handle.
+ #[test]
+ fn replace_is_visible_through_existing_clones() {
+ let store = UserScriptStore::new(Vec::new());
+ let handle = store.clone();
+ assert!(handle.is_empty());
+
+ store.replace(vec![compiled("added", "")]);
+
+ assert!(!handle.is_empty());
+ assert_eq!(
+ handle
+ .matching(&Url::parse("https://anything.test/").unwrap())
+ .len(),
+ 1
+ );
+ }
+}
diff --git a/privaxy/src/server/statistics.rs b/privaxy/src/server/statistics.rs
index d0654257..133984b0 100644
--- a/privaxy/src/server/statistics.rs
+++ b/privaxy/src/server/statistics.rs
@@ -1,3 +1,4 @@
+use portable_atomic::{AtomicU64, Ordering};
use serde::Serialize;
use std::{
collections::HashMap,
@@ -21,9 +22,9 @@ pub struct SerializableStatistics {
#[derive(Debug, Clone)]
pub struct Statistics {
- pub proxied_requests: Arc>,
- pub blocked_requests: Arc>,
- pub modified_responses: Arc>,
+ pub proxied_requests: Arc,
+ pub blocked_requests: Arc,
+ pub modified_responses: Arc,
pub top_blocked_paths: Arc>>,
pub top_clients: Arc>>,
}
@@ -37,9 +38,9 @@ impl Default for Statistics {
impl Statistics {
pub fn new() -> Self {
Self {
- proxied_requests: Arc::new(Mutex::new(0)),
- blocked_requests: Arc::new(Mutex::new(0)),
- modified_responses: Arc::new(Mutex::new(0)),
+ proxied_requests: Arc::new(AtomicU64::new(0)),
+ blocked_requests: Arc::new(AtomicU64::new(0)),
+ modified_responses: Arc::new(AtomicU64::new(0)),
top_blocked_paths: Arc::new(Mutex::new(LRUCache::default())),
top_clients: Arc::new(Mutex::new(HashMap::new())),
}
@@ -63,31 +64,22 @@ impl Statistics {
}
pub fn increment_proxied_requests(&self) -> u64 {
- let mut proxied_requests = self.proxied_requests.lock().unwrap();
-
- *proxied_requests += 1;
- *proxied_requests
+ self.proxied_requests.fetch_add(1, Ordering::Relaxed) + 1
}
pub fn increment_blocked_requests(&self) -> u64 {
- let mut blocked_requests = self.blocked_requests.lock().unwrap();
-
- *blocked_requests += 1;
- *blocked_requests
+ self.blocked_requests.fetch_add(1, Ordering::Relaxed) + 1
}
pub fn increment_modified_responses(&self) -> u64 {
- let mut modified_responses = self.modified_responses.lock().unwrap();
-
- *modified_responses += 1;
- *modified_responses
+ self.modified_responses.fetch_add(1, Ordering::Relaxed) + 1
}
pub fn get_serialized(&self) -> SerializableStatistics {
SerializableStatistics {
- proxied_requests: *self.proxied_requests.lock().unwrap(),
- blocked_requests: *self.blocked_requests.lock().unwrap(),
- modified_responses: *self.modified_responses.lock().unwrap(),
+ proxied_requests: self.proxied_requests.load(Ordering::Relaxed),
+ blocked_requests: self.blocked_requests.load(Ordering::Relaxed),
+ modified_responses: self.modified_responses.load(Ordering::Relaxed),
top_blocked_paths: {
let top_blocked_paths = self.top_blocked_paths.lock().unwrap();
let mut top_blocked_paths_iterator = top_blocked_paths.iter();
diff --git a/privaxy/src/server/web_gui/mod.rs b/privaxy/src/server/web_gui/mod.rs
index 65586d38..0c865253 100644
--- a/privaxy/src/server/web_gui/mod.rs
+++ b/privaxy/src/server/web_gui/mod.rs
@@ -1,7 +1,9 @@
use crate::configuration::FilterFailureStore;
use crate::logging::LogHandle;
use crate::proxy::exclusions::LocalExclusionStore;
+use crate::proxy::gm::storage::GmStorageStore;
use crate::proxy::tls_failures::TlsFailureStore;
+use crate::proxy::userscripts::{PrivateNetworkAccess, UserScriptStore};
use crate::statistics::Statistics;
use crate::WEBAPP_FRONTEND_DIR;
use crate::{blocker::BlockingDisabledStore, configuration::Configuration};
@@ -26,6 +28,7 @@ mod pac;
pub(crate) mod settings;
pub(crate) mod statistics;
pub(crate) mod tls_failures;
+pub(crate) mod userscripts;
#[derive(Debug, Serialize)]
pub(crate) struct ApiError {
@@ -41,6 +44,9 @@ pub(crate) fn get_frontend(
local_exclusions_store: &LocalExclusionStore,
tls_failure_store: &TlsFailureStore,
filter_failure_store: &FilterFailureStore,
+ user_script_store: &UserScriptStore,
+ gm_storage: &GmStorageStore,
+ private_network_access: &PrivateNetworkAccess,
notify_reload: Arc,
log_handle: LogHandle,
) -> BoxedFilter<(impl warp::Reply,)> {
@@ -66,6 +72,9 @@ pub(crate) fn get_frontend(
local_exclusions_store,
tls_failure_store,
filter_failure_store,
+ user_script_store,
+ gm_storage,
+ private_network_access,
http_client,
notify_reload,
log_handle,
@@ -119,6 +128,9 @@ fn create_api_routes(
local_exclusions_store: &LocalExclusionStore,
tls_failure_store: &TlsFailureStore,
filter_failure_store: &FilterFailureStore,
+ user_script_store: &UserScriptStore,
+ gm_storage: &GmStorageStore,
+ private_network_access: &PrivateNetworkAccess,
http_client: reqwest::Client,
notify_reload: Arc,
log_handle: LogHandle,
@@ -217,6 +229,17 @@ fn create_api_routes(
.and(require_auth.clone())
.and(filterlists::create_routes());
+ let userscripts_route =
+ warp::path("userscripts")
+ .and(require_auth.clone())
+ .and(userscripts::create_routes(
+ configuration_save_lock.clone(),
+ http_client.clone(),
+ user_script_store.clone(),
+ gm_storage.clone(),
+ private_network_access.clone(),
+ ));
+
// Note: `.recover` is attached to the inner combinator, NOT to the
// outer `api_path.and(...)`. If we attached it to the outer filter,
// recover would also fire when `api_path` itself didn't match (e.g.
@@ -234,6 +257,7 @@ fn create_api_routes(
.or(settings_route)
.or(options_route)
.or(filterlists_route)
+ .or(userscripts_route)
.recover(handle_rejection);
api_path.and(api_inner).with(def_headers).boxed()
diff --git a/privaxy/src/server/web_gui/userscripts.rs b/privaxy/src/server/web_gui/userscripts.rs
new file mode 100644
index 00000000..6e3fad2b
--- /dev/null
+++ b/privaxy/src/server/web_gui/userscripts.rs
@@ -0,0 +1,594 @@
+//! `/api/userscripts` — install, edit, toggle and remove userscripts at
+//! runtime.
+//!
+//! Every mutation persists the configuration and then replaces the contents of
+//! the [`UserScriptStore`], so a change made in the web UI applies to the next
+//! page load without a proxy reload.
+//!
+//! Unlike the filter routes, these deliberately do *not* push through
+//! `configuration_updater_sender`: that channel makes the updater re-read every
+//! filter list from disk and rebuild the adblock engine, which a userscript
+//! change has no reason to trigger. The updater's own configuration copy only
+//! ever touches filters, so leaving it untouched is safe.
+
+use super::{get_error_response, get_unprocessable_response};
+use crate::configuration::{
+ calc_local_userscript_filename, calc_userscript_filename, fetch_userscript, Configuration,
+ ConfigurationError, RefreshOutcome, UserScript, UserScriptMetadata, UserScriptRefresh,
+ UserScriptUpdate,
+};
+use crate::proxy::gm::storage::GmStorageStore;
+use crate::proxy::userscripts::{reload_userscripts, PrivateNetworkAccess, UserScriptStore};
+use serde::{Deserialize, Serialize};
+use std::{convert::Infallible, sync::Arc};
+use url::Url;
+use warp::filters::BoxedFilter;
+use warp::http::Response;
+use warp::Filter;
+
+/// The userscript section as served to the web UI: the master switch plus every
+/// installed script.
+#[derive(Debug, Serialize)]
+struct UserScriptsResponse {
+ enabled: bool,
+ allow_private_network_requests: bool,
+ scripts: Vec,
+}
+
+/// An installed script, annotated with what its metadata block declares so the
+/// UI can show where it runs without parsing JavaScript itself.
+#[derive(Debug, Serialize)]
+struct UserScriptResponse {
+ enabled: bool,
+ title: String,
+ file_name: String,
+ url: Option,
+ version: Option,
+ description: Option,
+ run_at: Option,
+ matches: Vec,
+ grants: Vec,
+ no_frames: bool,
+ /// Set when the stored body could not be read or no longer parses. Such a
+ /// script is skipped at injection time, so the UI must be able to say so.
+ error: Option,
+ /// Non-fatal compile problems, e.g. a `@require` library that could not be
+ /// fetched. The script still runs, degraded.
+ warnings: Vec,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct UserScriptStatusChangeRequest {
+ file_name: String,
+ enabled: bool,
+}
+
+/// `PUT /api/userscripts/enabled` — the engine-level switches. Both fields are
+/// optional so a client may change one without knowing the other's value.
+#[derive(Debug, Deserialize)]
+pub struct EngineSettingsRequest {
+ #[serde(default)]
+ enabled: Option,
+ #[serde(default)]
+ allow_private_network_requests: Option,
+}
+
+/// Install a script, either from pasted source or by fetching a URL.
+#[derive(Debug, Deserialize)]
+pub struct AddUserScriptRequest {
+ /// Script source, pasted directly into the UI.
+ #[serde(default)]
+ body: Option,
+ /// URL to fetch the script from, for installing from e.g. Greasyfork.
+ #[serde(default)]
+ url: Option,
+ #[serde(default = "enabled_by_default")]
+ enabled: bool,
+}
+
+fn enabled_by_default() -> bool {
+ true
+}
+
+#[derive(Debug, Deserialize)]
+pub struct UpdateUserScriptRequest {
+ file_name: String,
+ body: String,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct DeleteUserScriptRequest {
+ file_name: String,
+}
+
+/// `POST /api/userscripts/update` — refresh from upstream on demand rather than
+/// waiting out the 24h timer.
+#[derive(Debug, Deserialize)]
+pub struct RefreshRequest {
+ /// One script, or every URL-installed script when absent.
+ #[serde(default)]
+ file_name: Option,
+}
+
+/// Turn a stored script plus its on-disk body into the UI representation,
+/// reporting rather than hiding a body that cannot be read or parsed.
+///
+/// Compile warnings (an unreachable `@require`, say) are read from the live
+/// store rather than recomputed, so the UI reports exactly what the injection
+/// path is actually working with. A disabled script is absent from the store and
+/// therefore has no warnings — it isn't being injected at all.
+async fn describe_script(
+ script: &UserScript,
+ user_script_store: &UserScriptStore,
+) -> UserScriptResponse {
+ let mut response = UserScriptResponse {
+ enabled: script.enabled,
+ title: script.title.clone(),
+ file_name: script.file_name.clone(),
+ url: script.url.as_ref().map(|url| url.to_string()),
+ version: None,
+ description: None,
+ run_at: None,
+ matches: Vec::new(),
+ grants: Vec::new(),
+ no_frames: false,
+ error: None,
+ warnings: user_script_store
+ .find(&script.file_name)
+ .map(|compiled| compiled.warnings.clone())
+ .unwrap_or_default(),
+ };
+
+ let body = match script.read_body().await {
+ Ok(body) => body,
+ Err(err) => {
+ response.error = Some(format!("Unable to read the script body: {err}"));
+ return response;
+ }
+ };
+
+ match UserScriptMetadata::parse(&body) {
+ Ok(metadata) => {
+ response.version = metadata.version.clone();
+ response.description = metadata.description.clone();
+ response.run_at = Some(metadata.run_at.as_token().to_string());
+ response.no_frames = metadata.no_frames;
+ response.grants = metadata.grants.clone();
+ response.matches = metadata
+ .matches
+ .iter()
+ .map(|pattern| pattern.as_str().to_string())
+ .chain(
+ metadata
+ .includes
+ .iter()
+ .map(|pattern| pattern.as_str().to_string()),
+ )
+ .collect();
+ }
+ Err(err) => response.error = Some(err.to_string()),
+ }
+
+ response
+}
+
+async fn get_userscripts(
+ user_script_store: UserScriptStore,
+) -> Result {
+ let configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => {
+ log::error!("Failed to read configuration: {err}");
+ return Ok(get_error_response(err));
+ }
+ };
+
+ let mut scripts = Vec::with_capacity(configuration.userscripts.scripts.len());
+ for script in &configuration.userscripts.scripts {
+ scripts.push(describe_script(script, &user_script_store).await);
+ }
+
+ let response = UserScriptsResponse {
+ enabled: configuration.userscripts.enabled,
+ allow_private_network_requests: configuration.userscripts.allow_private_network_requests,
+ scripts,
+ };
+
+ Ok(Response::builder()
+ .header(http::header::CONTENT_TYPE, "application/json")
+ .body(serde_json::to_string(&response).unwrap())
+ .unwrap())
+}
+
+async fn get_userscript_body(
+ file_name: String,
+ _user_script_store: UserScriptStore,
+) -> Result {
+ let configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => {
+ log::error!("Failed to read configuration: {err}");
+ return Ok(get_error_response(err));
+ }
+ };
+
+ let Some(script) = configuration.userscripts.find(&file_name) else {
+ return Ok(super::get_unprocessable_response("No such userscript"));
+ };
+
+ match script.read_body().await {
+ Ok(body) => Ok(Response::builder()
+ .header(http::header::CONTENT_TYPE, "text/plain; charset=utf-8")
+ .body(body)
+ .unwrap()),
+ Err(err) => {
+ log::error!("Failed to read userscript body: {err}");
+ Ok(get_error_response(err))
+ }
+ }
+}
+
+/// Map a configuration error to the right status: a rejected script is the
+/// caller's problem (422), anything else is ours (500).
+fn userscript_error_response(err: ConfigurationError) -> Response {
+ match err {
+ ConfigurationError::UserScript(err) => get_unprocessable_response(&err.to_string()),
+ ConfigurationError::UserScriptFetchError(message) => get_unprocessable_response(&message),
+ ConfigurationError::UserScriptNotFound(_) => {
+ get_unprocessable_response("No such userscript")
+ }
+ err => {
+ log::error!("Userscript operation failed: {err}");
+ get_error_response(err)
+ }
+ }
+}
+
+async fn set_engine_settings(
+ request: EngineSettingsRequest,
+ http_client: reqwest::Client,
+ configuration_save_lock: Arc>,
+ user_script_store: UserScriptStore,
+ private_network_access: PrivateNetworkAccess,
+) -> Result {
+ // The lock is held across read-modify-write so a concurrent mutation cannot
+ // be lost by writing back a configuration read before it landed.
+ let _guard = configuration_save_lock.lock().await;
+
+ let mut configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => return Ok(get_error_response(err)),
+ };
+
+ if let Some(enabled) = request.enabled {
+ configuration.userscripts.enabled = enabled;
+ }
+ if let Some(allowed) = request.allow_private_network_requests {
+ configuration.userscripts.allow_private_network_requests = allowed;
+ }
+
+ if let Err(err) = configuration.save().await {
+ return Ok(userscript_error_response(err));
+ }
+
+ // Applied to the live relay switch immediately, so this behaves like every
+ // other userscript setting rather than waiting for a reload.
+ private_network_access.set(configuration.userscripts.allow_private_network_requests);
+
+ reload_userscripts(&user_script_store, &configuration, &http_client).await;
+
+ Ok(Response::builder()
+ .status(http::StatusCode::ACCEPTED)
+ .body(String::new())
+ .unwrap())
+}
+
+async fn change_userscript_status(
+ requests: Vec,
+ http_client: reqwest::Client,
+ configuration_save_lock: Arc>,
+ user_script_store: UserScriptStore,
+) -> Result {
+ let _guard = configuration_save_lock.lock().await;
+
+ let mut configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => return Ok(get_error_response(err)),
+ };
+
+ for request in requests {
+ if let Err(err) = configuration
+ .set_userscript_enabled_status(&request.file_name, request.enabled)
+ .await
+ {
+ return Ok(userscript_error_response(err));
+ }
+ }
+
+ reload_userscripts(&user_script_store, &configuration, &http_client).await;
+
+ Ok(Response::builder()
+ .status(http::StatusCode::ACCEPTED)
+ .body(String::new())
+ .unwrap())
+}
+
+async fn add_userscript(
+ request: AddUserScriptRequest,
+ http_client: reqwest::Client,
+ configuration_save_lock: Arc>,
+ user_script_store: UserScriptStore,
+) -> Result {
+ // Resolve the source before taking the lock: fetching a remote script can
+ // be slow and must not serialize other configuration writers.
+ let (body, url) = match (request.body, request.url) {
+ (_, Some(url)) => {
+ let Ok(url) = Url::parse(url.trim()) else {
+ return Ok(get_unprocessable_response("The script URL is not valid"));
+ };
+ match fetch_userscript(&url, &http_client).await {
+ Ok(body) => (body, Some(url)),
+ Err(err) => return Ok(userscript_error_response(err)),
+ }
+ }
+ (Some(body), None) => (body, None),
+ (None, None) => {
+ return Ok(get_unprocessable_response(
+ "Provide either a script body or a URL to install from",
+ ))
+ }
+ };
+
+ let file_name = match &url {
+ // Keyed by URL so re-installing the same script reuses its body file.
+ Some(url) => calc_userscript_filename(url.as_str()),
+ None => calc_local_userscript_filename(),
+ };
+
+ let _guard = configuration_save_lock.lock().await;
+
+ let mut configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => return Ok(get_error_response(err)),
+ };
+
+ if configuration.userscripts.find(&file_name).is_some() {
+ return Ok(Response::builder()
+ .status(http::StatusCode::CONFLICT)
+ .header(http::header::CONTENT_TYPE, "application/json")
+ .body(
+ serde_json::to_string(&super::ApiError {
+ error: "This userscript is already installed".to_string(),
+ })
+ .unwrap(),
+ )
+ .unwrap());
+ }
+
+ let script = UserScript {
+ enabled: request.enabled,
+ // Replaced with the parsed `@name` by `add_userscript`.
+ title: String::new(),
+ file_name,
+ url,
+ };
+
+ if let Err(err) = configuration.add_userscript(script, &body).await {
+ return Ok(userscript_error_response(err));
+ }
+
+ reload_userscripts(&user_script_store, &configuration, &http_client).await;
+
+ Ok(Response::builder()
+ .status(http::StatusCode::CREATED)
+ .body(String::new())
+ .unwrap())
+}
+
+/// Refresh one script or all of them from upstream.
+///
+/// Unlike the periodic refresh in `ConfigurationUpdater`, this holds the save
+/// lock, so a `@name` or `@version` that changed upstream is persisted rather
+/// than being recomputed on every read.
+async fn refresh_userscripts(
+ request: RefreshRequest,
+ http_client: reqwest::Client,
+ configuration_save_lock: Arc>,
+ user_script_store: UserScriptStore,
+) -> Result {
+ let _guard = configuration_save_lock.lock().await;
+
+ let mut configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => return Ok(get_error_response(err)),
+ };
+
+ let outcomes = match request.file_name {
+ Some(file_name) => {
+ let Some(script) = configuration
+ .userscripts
+ .scripts
+ .iter_mut()
+ .find(|script| script.file_name == file_name)
+ else {
+ return Ok(get_unprocessable_response("No such userscript"));
+ };
+
+ if script.url.is_none() {
+ return Ok(get_unprocessable_response(
+ "This userscript was pasted rather than installed from a URL, so there is \
+ nothing to refresh.",
+ ));
+ }
+
+ let outcome = match script.update(&http_client).await {
+ Ok(UserScriptUpdate::Updated { version }) => RefreshOutcome::Updated { version },
+ Ok(UserScriptUpdate::AlreadyCurrent) => RefreshOutcome::AlreadyCurrent,
+ Err(err) => RefreshOutcome::Failed {
+ error: err.to_string(),
+ },
+ };
+
+ vec![UserScriptRefresh {
+ file_name: script.file_name.clone(),
+ title: script.title.clone(),
+ outcome,
+ }]
+ }
+ None => configuration.update_userscripts(&http_client).await,
+ };
+
+ // Titles and versions may have moved on; persist them while we hold the lock.
+ if let Err(err) = configuration.save().await {
+ return Ok(userscript_error_response(err));
+ }
+
+ reload_userscripts(&user_script_store, &configuration, &http_client).await;
+
+ Ok(Response::builder()
+ .header(http::header::CONTENT_TYPE, "application/json")
+ .body(serde_json::to_string(&outcomes).unwrap())
+ .unwrap())
+}
+
+async fn update_userscript(
+ request: UpdateUserScriptRequest,
+ http_client: reqwest::Client,
+ configuration_save_lock: Arc>,
+ user_script_store: UserScriptStore,
+) -> Result {
+ let _guard = configuration_save_lock.lock().await;
+
+ let mut configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => return Ok(get_error_response(err)),
+ };
+
+ if let Err(err) = configuration
+ .replace_userscript_body(&request.file_name, &request.body)
+ .await
+ {
+ return Ok(userscript_error_response(err));
+ }
+
+ reload_userscripts(&user_script_store, &configuration, &http_client).await;
+
+ Ok(Response::builder()
+ .status(http::StatusCode::OK)
+ .body(String::new())
+ .unwrap())
+}
+
+async fn delete_userscript(
+ request: DeleteUserScriptRequest,
+ http_client: reqwest::Client,
+ configuration_save_lock: Arc>,
+ user_script_store: UserScriptStore,
+ gm_storage: GmStorageStore,
+) -> Result {
+ let _guard = configuration_save_lock.lock().await;
+
+ let mut configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(err) => return Ok(get_error_response(err)),
+ };
+
+ if let Err(err) = configuration.remove_userscript(&request.file_name).await {
+ return Ok(userscript_error_response(err));
+ }
+
+ // An uninstalled script must not leave its stored values behind to be
+ // silently inherited if a script with the same file name is installed later.
+ gm_storage.forget(&request.file_name);
+
+ reload_userscripts(&user_script_store, &configuration, &http_client).await;
+
+ Ok(Response::builder()
+ .status(http::StatusCode::NO_CONTENT)
+ .body(String::new())
+ .unwrap())
+}
+
+pub(super) fn create_routes(
+ configuration_save_lock: Arc>,
+ http_client: reqwest::Client,
+ user_script_store: UserScriptStore,
+ gm_storage: GmStorageStore,
+ private_network_access: PrivateNetworkAccess,
+) -> BoxedFilter<(impl warp::Reply,)> {
+ // The master switch lives on a fixed sub-path, so it must be matched before
+ // the `` parameter route below would swallow it.
+ let engine_enabled_route = warp::path("enabled")
+ .and(warp::path::end())
+ .and(warp::put())
+ .and(warp::body::json())
+ .and(super::with_http_client(http_client.clone()))
+ .and(super::with_configuration_save_lock(
+ configuration_save_lock.clone(),
+ ))
+ .and(super::with_arc(user_script_store.clone()))
+ .and(super::with_arc(private_network_access))
+ .and_then(self::set_engine_settings);
+
+ let refresh_route = warp::path("update")
+ .and(warp::path::end())
+ .and(warp::post())
+ .and(warp::body::json())
+ .and(super::with_http_client(http_client.clone()))
+ .and(super::with_configuration_save_lock(
+ configuration_save_lock.clone(),
+ ))
+ .and(super::with_arc(user_script_store.clone()))
+ .and_then(self::refresh_userscripts);
+
+ let body_route = warp::path::param::()
+ .and(warp::path("body"))
+ .and(warp::path::end())
+ .and(warp::get())
+ .and(super::with_arc(user_script_store.clone()))
+ .and_then(self::get_userscript_body);
+
+ engine_enabled_route
+ .or(refresh_route)
+ .or(body_route)
+ .or(warp::path::end()
+ .and(warp::get())
+ .and(super::with_arc(user_script_store.clone()))
+ .and_then(self::get_userscripts))
+ .or(warp::path::end()
+ .and(warp::put())
+ .and(warp::body::json())
+ .and(super::with_http_client(http_client.clone()))
+ .and(super::with_configuration_save_lock(
+ configuration_save_lock.clone(),
+ ))
+ .and(super::with_arc(user_script_store.clone()))
+ .and_then(self::change_userscript_status))
+ .or(warp::path::end()
+ .and(warp::post())
+ .and(warp::body::json())
+ .and(super::with_http_client(http_client.clone()))
+ .and(super::with_configuration_save_lock(
+ configuration_save_lock.clone(),
+ ))
+ .and(super::with_arc(user_script_store.clone()))
+ .and_then(self::add_userscript))
+ .or(warp::path::end()
+ .and(warp::patch())
+ .and(warp::body::json())
+ .and(super::with_http_client(http_client.clone()))
+ .and(super::with_configuration_save_lock(
+ configuration_save_lock.clone(),
+ ))
+ .and(super::with_arc(user_script_store.clone()))
+ .and_then(self::update_userscript))
+ .or(warp::path::end()
+ .and(warp::delete())
+ .and(warp::body::json())
+ .and(super::with_http_client(http_client))
+ .and(super::with_configuration_save_lock(configuration_save_lock))
+ .and(super::with_arc(user_script_store))
+ .and(super::with_arc(gm_storage))
+ .and_then(self::delete_userscript))
+ .boxed()
+}
diff --git a/web_frontend/src/main.rs b/web_frontend/src/main.rs
index dcc34317..979aaef1 100644
--- a/web_frontend/src/main.rs
+++ b/web_frontend/src/main.rs
@@ -26,6 +26,8 @@ mod settings;
mod settings_textarea;
mod submit_banner;
mod tls_failures;
+mod userscript_edit;
+mod userscripts;
#[derive(Debug, Deserialize, Clone)]
pub(crate) struct ApiError {
diff --git a/web_frontend/src/settings.rs b/web_frontend/src/settings.rs
index 0fd78c01..001114d5 100644
--- a/web_frontend/src/settings.rs
+++ b/web_frontend/src/settings.rs
@@ -6,6 +6,7 @@ use crate::general::GeneralSettings;
use crate::pac::PacSettingsPage;
use crate::set_title;
use crate::settings_textarea::SettingsTextarea;
+use crate::userscripts::UserScriptsPage;
use yew::prelude::*;
use yew::{html, Html};
use yew_router::prelude::*;
@@ -20,6 +21,8 @@ pub enum SettingsRoute {
Exclusions,
#[at("/settings/custom-filters")]
CustomFilters,
+ #[at("/settings/userscripts")]
+ Userscripts,
#[at("/settings/pac")]
Pac,
#[at("/settings/account")]
@@ -89,6 +92,11 @@ pub fn switch_settings(route: SettingsRoute) -> Html {
html! {}
}
+ SettingsRoute::Userscripts => {
+ set_title("Settings - Userscripts");
+
+ html! { }
+ }
SettingsRoute::Pac => {
set_title("Settings - PAC");
@@ -112,6 +120,7 @@ pub fn switch_settings(route: SettingsRoute) -> Html {
classes={get_classes(route, SettingsRoute::Filters)} to={SettingsRoute::Filters}> { "Filters" }>
classes={get_classes(route, SettingsRoute::Exclusions)} to={SettingsRoute::Exclusions}> { "Exclusions" }>
classes={get_classes(route, SettingsRoute::CustomFilters)} to={SettingsRoute::CustomFilters}> { "Custom filters" }>
+ classes={get_classes(route, SettingsRoute::Userscripts)} to={SettingsRoute::Userscripts}> { "Userscripts" }>
classes={get_classes(route, SettingsRoute::Pac)} to={SettingsRoute::Pac}> { "PAC" }>
classes={get_classes(route, SettingsRoute::Account)} to={SettingsRoute::Account}> { "Account" }>
classes={get_classes(route, SettingsRoute::Debug)} to={SettingsRoute::Debug}> { "Debug" }>
diff --git a/web_frontend/src/userscript_edit.rs b/web_frontend/src/userscript_edit.rs
new file mode 100644
index 00000000..cc0ee653
--- /dev/null
+++ b/web_frontend/src/userscript_edit.rs
@@ -0,0 +1,280 @@
+//! Modal for editing or uninstalling a userscript.
+//!
+//! The body is fetched on open rather than passed in as a property: script
+//! bodies are large and the list endpoint deliberately serves only metadata.
+
+use crate::{failure_banner, ApiError};
+use gloo_net::http::Request;
+use serde::Serialize;
+use wasm_bindgen_futures::spawn_local;
+use web_sys::HtmlTextAreaElement;
+use yew::prelude::*;
+
+const USERSCRIPTS_RESOURCE_URL: &str = "/api/userscripts";
+
+#[derive(Debug, Serialize)]
+struct UpdateUserScriptRequest {
+ file_name: String,
+ body: String,
+}
+
+#[derive(Debug, Serialize)]
+struct DeleteUserScriptRequest {
+ file_name: String,
+}
+
+#[derive(Properties, PartialEq)]
+pub struct Props {
+ /// Identifies the script to the backend; stable across edits, including
+ /// edits that change its `@name`.
+ pub file_name: String,
+ pub title: String,
+ /// Emitted when the modal is dismissed without a change.
+ pub on_close: Callback<()>,
+ /// Emitted after a successful save or delete; the parent should unmount the
+ /// modal and reload its script list.
+ pub on_changed: Callback<()>,
+}
+
+pub enum Message {
+ BodyLoaded(String),
+ BodyChanged(String),
+ Save,
+ Delete,
+ ConfirmDelete,
+ CancelDelete,
+ Succeeded,
+ Failed(String),
+ AcknowledgeError,
+}
+
+pub struct UserScriptEditModal {
+ body: Option,
+ busy: bool,
+ confirming_delete: bool,
+ error: Option,
+}
+
+impl Component for UserScriptEditModal {
+ type Message = Message;
+ type Properties = Props;
+
+ fn create(ctx: &Context) -> Self {
+ Self::fetch_body(ctx);
+
+ Self {
+ body: None,
+ busy: false,
+ confirming_delete: false,
+ error: None,
+ }
+ }
+
+ fn changed(&mut self, ctx: &Context, old_props: &Self::Properties) -> bool {
+ // Re-fetch when the modal is pointed at a different script without
+ // being unmounted in between.
+ if ctx.props().file_name != old_props.file_name {
+ self.body = None;
+ self.busy = false;
+ self.confirming_delete = false;
+ self.error = None;
+ Self::fetch_body(ctx);
+ }
+ true
+ }
+
+ fn update(&mut self, ctx: &Context, msg: Self::Message) -> bool {
+ match msg {
+ Message::BodyLoaded(body) => self.body = Some(body),
+ Message::BodyChanged(body) => self.body = Some(body),
+ Message::Save => {
+ if self.busy {
+ return false;
+ }
+ let Some(body) = self.body.clone() else {
+ return false;
+ };
+
+ let request_body = serde_json::to_string(&UpdateUserScriptRequest {
+ file_name: ctx.props().file_name.clone(),
+ body,
+ })
+ .unwrap();
+
+ self.dispatch(ctx, Request::patch(USERSCRIPTS_RESOURCE_URL), request_body);
+ }
+ Message::Delete => self.confirming_delete = true,
+ Message::CancelDelete => self.confirming_delete = false,
+ Message::ConfirmDelete => {
+ if self.busy {
+ return false;
+ }
+
+ let request_body = serde_json::to_string(&DeleteUserScriptRequest {
+ file_name: ctx.props().file_name.clone(),
+ })
+ .unwrap();
+
+ self.dispatch(ctx, Request::delete(USERSCRIPTS_RESOURCE_URL), request_body);
+ }
+ Message::Succeeded => {
+ self.busy = false;
+ ctx.props().on_changed.emit(());
+ }
+ Message::Failed(message) => {
+ log::error!("Userscript edit failed: {message}");
+ self.busy = false;
+ self.confirming_delete = false;
+ self.error = Some(message);
+ }
+ Message::AcknowledgeError => self.error = None,
+ }
+
+ true
+ }
+
+ fn view(&self, ctx: &Context) -> Html {
+ let on_body_input = ctx.link().callback(|event: InputEvent| {
+ let textarea: HtmlTextAreaElement = event.target_unchecked_into();
+ Message::BodyChanged(textarea.value())
+ });
+
+ let error_banner = match &self.error {
+ Some(message) => failure_banner!(
+ true,
+ ctx.link().callback(|_| Message::AcknowledgeError),
+ message.clone()
+ ),
+ None => html! {},
+ };
+
+ let body_editor = match &self.body {
+ Some(body) => html! {
+
+ },
+ None => html! {
{ "Loading script..." }
},
+ };
+
+ html! {
+
+
+
{ &ctx.props().title }
+
+ { error_banner }
+
+
+ { body_editor }
+
+ { "Saving re-reads the metadata block, so changes to " }
+ { "@match" }
+ { " or " }
+ { "@name" }
+ { " take effect on the next page load." }
+
+
+ { self.render_actions(ctx) }
+
+
+
+ }
+ }
+}
+
+impl UserScriptEditModal {
+ fn fetch_body(ctx: &Context) {
+ let link = ctx.link().clone();
+ let file_name = ctx.props().file_name.clone();
+
+ spawn_local(async move {
+ let url = format!("{USERSCRIPTS_RESOURCE_URL}/{file_name}/body");
+ match Request::get(&url).send().await {
+ Ok(response) if response.ok() => match response.text().await {
+ Ok(body) => link.send_message(Message::BodyLoaded(body)),
+ Err(err) => link.send_message(Message::Failed(format!(
+ "Unable to read the script body: {err}"
+ ))),
+ },
+ Ok(response) => link.send_message(Message::Failed(format!(
+ "Unable to load the script body (HTTP {})",
+ response.status()
+ ))),
+ Err(err) => link.send_message(Message::Failed(format!("{err}"))),
+ }
+ });
+ }
+
+ /// Send a mutating request, mapping a non-2xx JSON body onto the error
+ /// banner so validation failures (a script with no `@match`, say) are shown
+ /// verbatim.
+ fn dispatch(
+ &mut self,
+ ctx: &Context,
+ builder: gloo_net::http::RequestBuilder,
+ body: String,
+ ) {
+ self.busy = true;
+ self.error = None;
+
+ let link = ctx.link().clone();
+ let request = builder
+ .header("Content-Type", "application/json")
+ .body(body)
+ .unwrap();
+
+ spawn_local(async move {
+ match request.send().await {
+ Ok(response) if response.ok() => link.send_message(Message::Succeeded),
+ Ok(response) => {
+ let status = response.status();
+ let message = match response.json::().await {
+ Ok(api_error) => api_error.error,
+ Err(_) => format!("Request failed (HTTP {status})"),
+ };
+ link.send_message(Message::Failed(message));
+ }
+ Err(err) => link.send_message(Message::Failed(format!("{err}"))),
+ }
+ });
+ }
+
+ fn render_actions(&self, ctx: &Context) -> Html {
+ if self.confirming_delete {
+ return html! {
+
+ }
+ }
+}
diff --git a/web_frontend/src/userscripts.rs b/web_frontend/src/userscripts.rs
new file mode 100644
index 00000000..e76c7bf7
--- /dev/null
+++ b/web_frontend/src/userscripts.rs
@@ -0,0 +1,757 @@
+use crate::userscript_edit::UserScriptEditModal;
+use crate::{failure_banner, submit_banner, ApiError};
+use gloo_net::http::Request;
+use serde::{Deserialize, Serialize};
+use wasm_bindgen_futures::spawn_local;
+use web_sys::{HtmlInputElement, HtmlTextAreaElement};
+use yew::prelude::*;
+use yew::{html, Component, Context, Html};
+
+/// One installed userscript, as served by `GET /api/userscripts`.
+#[derive(Debug, Clone, Deserialize, PartialEq, Eq)]
+pub struct UserScript {
+ pub enabled: bool,
+ pub title: String,
+ pub file_name: String,
+ pub url: Option,
+ pub version: Option,
+ pub description: Option,
+ pub run_at: Option,
+ pub matches: Vec,
+ pub grants: Vec,
+ pub no_frames: bool,
+ /// Set when the stored body cannot be read or no longer parses; such a
+ /// script is skipped at injection time.
+ pub error: Option,
+ /// Non-fatal problems, e.g. a `@require` library that could not be fetched.
+ /// The script still runs, degraded.
+ #[serde(default)]
+ pub warnings: Vec,
+}
+
+/// The `[userscripts]` configuration section, as served by the API.
+#[derive(Debug, Clone, Deserialize, PartialEq, Eq)]
+pub struct UserScriptsConfig {
+ pub enabled: bool,
+ #[serde(default)]
+ pub allow_private_network_requests: bool,
+ pub scripts: Vec,
+}
+
+/// Both fields optional so each toggle sends only what it changed.
+#[derive(Debug, Default, Serialize)]
+struct EngineSettingsRequest {
+ #[serde(skip_serializing_if = "Option::is_none")]
+ enabled: Option,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ allow_private_network_requests: Option,
+}
+
+#[derive(Debug, Serialize)]
+struct StatusChangeRequest {
+ file_name: String,
+ enabled: bool,
+}
+
+#[derive(Debug, Serialize)]
+struct RefreshRequest {
+ /// Absent means "every URL-installed script".
+ #[serde(skip_serializing_if = "Option::is_none")]
+ file_name: Option,
+}
+
+/// One entry of the refresh report.
+#[derive(Debug, Clone, Deserialize, PartialEq, Eq)]
+pub struct RefreshResult {
+ title: String,
+ outcome: String,
+ #[serde(default)]
+ version: Option,
+ #[serde(default)]
+ error: Option,
+}
+
+#[derive(Debug, Serialize)]
+struct AddUserScriptRequest {
+ #[serde(skip_serializing_if = "Option::is_none")]
+ body: Option,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ url: Option,
+ enabled: bool,
+}
+
+pub enum Message {
+ Load,
+ Loaded(UserScriptsConfig),
+ LoadFailed(String),
+ ToggleEngine(bool),
+ TogglePrivateNetwork(bool),
+ ToggleScript(String, bool),
+ OpenAdd,
+ CloseAdd,
+ AddUrlChanged(String),
+ AddBodyChanged(String),
+ SubmitAdd,
+ RefreshAll,
+ Refreshed(Vec),
+ Saved(&'static str),
+ Failed(String),
+ AckError,
+ AckSaved,
+ OpenEdit(Box),
+ CloseEdit,
+ EditCompleted,
+}
+
+pub struct UserScriptsPage {
+ config: Option,
+ add_open: bool,
+ add_url: String,
+ add_body: String,
+ submitting: bool,
+ saved_message: Option<&'static str>,
+ error: Option,
+ editing: Option,
+ /// Per-script outcome of the last on-demand refresh.
+ refresh_report: Option>,
+}
+
+impl Component for UserScriptsPage {
+ type Message = Message;
+ type Properties = ();
+
+ fn create(ctx: &Context) -> Self {
+ ctx.link().send_message(Message::Load);
+
+ Self {
+ config: None,
+ add_open: false,
+ add_url: String::new(),
+ add_body: String::new(),
+ submitting: false,
+ saved_message: None,
+ error: None,
+ editing: None,
+ refresh_report: None,
+ }
+ }
+
+ fn update(&mut self, ctx: &Context, msg: Self::Message) -> bool {
+ match msg {
+ Message::Load => {
+ let link = ctx.link().clone();
+ spawn_local(async move {
+ match Request::get("/api/userscripts").send().await {
+ Ok(response) if response.ok() => {
+ match response.json::().await {
+ Ok(config) => link.send_message(Message::Loaded(config)),
+ Err(err) => link.send_message(Message::LoadFailed(format!(
+ "Unable to read the userscript list: {err}"
+ ))),
+ }
+ }
+ Ok(response) => link.send_message(Message::LoadFailed(format!(
+ "Unable to load userscripts (HTTP {})",
+ response.status()
+ ))),
+ Err(err) => link.send_message(Message::LoadFailed(format!("{err}"))),
+ }
+ });
+ }
+ Message::Loaded(config) => {
+ self.config = Some(config);
+ self.submitting = false;
+ }
+ Message::LoadFailed(message) => {
+ log::error!("{message}");
+ self.error = Some(message);
+ self.submitting = false;
+ }
+ Message::ToggleEngine(enabled) => {
+ // Reflect the new value immediately; the request that follows
+ // both persists it and swaps the proxy's live script set, so it
+ // applies to the next page load.
+ if let Some(config) = self.config.as_mut() {
+ config.enabled = enabled;
+ }
+
+ let body = serde_json::to_string(&EngineSettingsRequest {
+ enabled: Some(enabled),
+ ..EngineSettingsRequest::default()
+ })
+ .unwrap();
+ self.send(
+ ctx,
+ Request::put("/api/userscripts/enabled"),
+ body,
+ "Userscript setting saved",
+ );
+ }
+ Message::TogglePrivateNetwork(allowed) => {
+ if let Some(config) = self.config.as_mut() {
+ config.allow_private_network_requests = allowed;
+ }
+
+ let body = serde_json::to_string(&EngineSettingsRequest {
+ allow_private_network_requests: Some(allowed),
+ ..EngineSettingsRequest::default()
+ })
+ .unwrap();
+ self.send(
+ ctx,
+ Request::put("/api/userscripts/enabled"),
+ body,
+ "Userscript setting saved",
+ );
+ }
+ Message::ToggleScript(file_name, enabled) => {
+ if let Some(config) = self.config.as_mut() {
+ if let Some(script) = config
+ .scripts
+ .iter_mut()
+ .find(|script| script.file_name == file_name)
+ {
+ script.enabled = enabled;
+ }
+ }
+
+ let body = serde_json::to_string(&vec![StatusChangeRequest { file_name, enabled }])
+ .unwrap();
+ self.send(ctx, Request::put("/api/userscripts"), body, "Changes saved");
+ }
+ Message::OpenAdd => {
+ self.add_open = true;
+ self.add_url = String::new();
+ self.add_body = String::new();
+ self.error = None;
+ }
+ Message::CloseAdd => self.add_open = false,
+ Message::AddUrlChanged(url) => self.add_url = url,
+ Message::AddBodyChanged(body) => self.add_body = body,
+ Message::SubmitAdd => {
+ let url = self.add_url.trim().to_string();
+ let body = self.add_body.trim().to_string();
+
+ if url.is_empty() && body.is_empty() {
+ self.error =
+ Some("Paste a script or provide a URL to install from.".to_string());
+ return true;
+ }
+
+ let request_body = serde_json::to_string(&AddUserScriptRequest {
+ body: if url.is_empty() { Some(body) } else { None },
+ url: if url.is_empty() { None } else { Some(url) },
+ enabled: true,
+ })
+ .unwrap();
+
+ self.send(
+ ctx,
+ Request::post("/api/userscripts"),
+ request_body,
+ "Userscript installed",
+ );
+ }
+ Message::RefreshAll => {
+ let body = serde_json::to_string(&RefreshRequest { file_name: None }).unwrap();
+ self.submitting = true;
+ self.error = None;
+
+ let link = ctx.link().clone();
+ let request = Request::post("/api/userscripts/update")
+ .header("Content-Type", "application/json")
+ .body(body)
+ .unwrap();
+
+ spawn_local(async move {
+ match request.send().await {
+ Ok(response) if response.ok() => {
+ match response.json::>().await {
+ Ok(results) => link.send_message(Message::Refreshed(results)),
+ Err(err) => link.send_message(Message::Failed(format!(
+ "Unable to read the refresh report: {err}"
+ ))),
+ }
+ }
+ Ok(response) => {
+ let status = response.status();
+ let message = match response.json::().await {
+ Ok(api_error) => api_error.error,
+ Err(_) => format!("Refresh failed (HTTP {status})"),
+ };
+ link.send_message(Message::Failed(message));
+ }
+ Err(err) => link.send_message(Message::Failed(format!("{err}"))),
+ }
+ });
+ }
+ Message::Refreshed(results) => {
+ self.submitting = false;
+ self.refresh_report = Some(results);
+ ctx.link().send_message(Message::Load);
+ }
+ Message::Saved(message) => {
+ self.submitting = false;
+ self.add_open = false;
+ self.error = None;
+ self.saved_message = Some(message);
+ ctx.link().send_message(Message::Load);
+ }
+ Message::Failed(message) => {
+ log::error!("Userscript request failed: {message}");
+ self.submitting = false;
+ self.error = Some(message);
+ // The optimistic toggle above may no longer reflect what was
+ // persisted, so re-read rather than leaving a stale checkbox.
+ ctx.link().send_message(Message::Load);
+ }
+ Message::AckError => self.error = None,
+ Message::AckSaved => self.saved_message = None,
+ Message::OpenEdit(script) => self.editing = Some(*script),
+ Message::CloseEdit => self.editing = None,
+ Message::EditCompleted => {
+ self.editing = None;
+ ctx.link().send_message(Message::Load);
+ }
+ }
+
+ true
+ }
+
+ fn view(&self, ctx: &Context) -> Html {
+ let title = html! {
+
+
{ "Userscripts" }
+
+ };
+
+ let Some(config) = &self.config else {
+ return html! { <>{ title }{ self.render_error(ctx) } };
+ };
+
+ html! {
+ <>
+ { title }
+ { self.render_scope_warning() }
+ { self.render_error(ctx) }
+ { self.render_saved_banner(ctx) }
+ { self.render_edit_modal(ctx) }
+ { self.render_engine_switch(ctx, config) }
+
+ { self.render_add_modal(ctx) }
+
+ }
+ }
+}
+
+impl UserScriptsPage {
+ /// Issue a mutating request and translate the outcome into a message.
+ fn send(
+ &mut self,
+ ctx: &Context,
+ builder: gloo_net::http::RequestBuilder,
+ body: String,
+ success_message: &'static str,
+ ) {
+ self.submitting = true;
+ self.error = None;
+
+ let link = ctx.link().clone();
+ let request = builder
+ .header("Content-Type", "application/json")
+ .body(body)
+ .unwrap();
+
+ spawn_local(async move {
+ match request.send().await {
+ Ok(response) if response.ok() => link.send_message(Message::Saved(success_message)),
+ Ok(response) => {
+ let status = response.status();
+ let message = match response.json::().await {
+ Ok(api_error) => api_error.error,
+ Err(_) => format!("Request failed (HTTP {status})"),
+ };
+ link.send_message(Message::Failed(message));
+ }
+ Err(err) => link.send_message(Message::Failed(format!("{err}"))),
+ }
+ });
+ }
+
+ /// Userscripts are arbitrary JavaScript injected into every matching page
+ /// for every client behind the proxy, which is a wider blast radius than a
+ /// browser extension installed in one profile. Say so where scripts are
+ /// added rather than burying it in documentation.
+ fn render_scope_warning(&self) -> Html {
+ html! {
+
+
+
+
+
{ "Userscripts run on every client behind this proxy" }
+
+ { "A script installed here is injected into every matching page on every device using Privaxy, and runs in the page's main world with full access to its contents. Only install scripts you have read and trust." }
+
+
+
+
+ }
+ }
+
+ fn render_error(&self, ctx: &Context) -> Html {
+ match &self.error {
+ Some(message) => failure_banner!(
+ true,
+ ctx.link().callback(|_| Message::AckError),
+ message.clone()
+ ),
+ None => html! {},
+ }
+ }
+
+ fn render_saved_banner(&self, ctx: &Context) -> Html {
+ let Some(message) = self.saved_message else {
+ return html! {};
+ };
+
+ let icon = html! {
+
+ };
+
+ html! {
+
+ }
+ }
+
+ fn render_engine_switch(&self, ctx: &Context, config: &UserScriptsConfig) -> Html {
+ let on_toggle = ctx.link().callback(|event: MouseEvent| {
+ let input: HtmlInputElement = event.target_unchecked_into();
+ Message::ToggleEngine(input.checked())
+ });
+
+ html! {
+
+ }
+ }
+
+ /// `GM_xmlhttpRequest` is relayed server-side, so it reaches the network the
+ /// proxy sits on rather than the browser's. Spell out what turning this on
+ /// exposes.
+ fn render_private_network_switch(
+ &self,
+ ctx: &Context,
+ config: &UserScriptsConfig,
+ ) -> Html {
+ let on_toggle = ctx.link().callback(|event: MouseEvent| {
+ let input: HtmlInputElement = event.target_unchecked_into();
+ Message::TogglePrivateNetwork(input.checked())
+ });
+
+ html! {
+
+
+
+ { "Allow requests to private addresses" }
+
+
+
+
+
+
+
+ { "Lets " }
+ { "GM_xmlhttpRequest" }
+ { " reach loopback, LAN and link-local addresses. Those requests are made by Privaxy, not by the browser, so they can reach routers, admin panels and cloud metadata endpoints that no page could contact. Leave off unless a script genuinely needs it." }
+
+
+
+ }
+ }
+
+ fn render_add_button(&self, ctx: &Context) -> Html {
+ html! {
+
+ }
+ }
+
+ fn render_refresh_button(&self, ctx: &Context) -> Html {
+ html! {
+
+ }
+ }
+
+ /// Report what the last refresh actually did, per script. Without this a
+ /// refresh that found nothing new is indistinguishable from one that failed.
+ fn render_refresh_report(&self) -> Html {
+ let Some(results) = &self.refresh_report else {
+ return html! {};
+ };
+
+ if results.is_empty() {
+ return html! {
+
+ { "No userscripts are installed from a URL, so there was nothing to check." }
+
+ };
+ }
+
+ html! {
+
+ { for results.iter().map(|result| {
+ let (classes, detail) = match result.outcome.as_str() {
+ "updated" => (
+ "text-green-700",
+ result.version.as_ref().map_or_else(
+ || "updated".to_string(),
+ |version| format!("updated to {version}"),
+ ),
+ ),
+ "already_current" => ("text-gray-500", "already up to date".to_string()),
+ _ => (
+ "text-red-600",
+ result.error.clone().unwrap_or_else(|| "failed".to_string()),
+ ),
+ };
+
+ html! {
+
+ }
+ if let Some((color, message)) = refresh_message {
+
{message}
+ }
{ render_category(FilterGroup::Default, filter_configuration) }
{ render_category(FilterGroup::Ads, filter_configuration) }
{ render_category(FilterGroup::Privacy, filter_configuration) }
From 31884036905d557c4c96f2ad185a7e3c5195235c Mon Sep 17 00:00:00 2001
From: Josh Mcdaniel <80354972+joshrmcdaniel@users.noreply.github.com>
Date: Wed, 30 Sep 2026 21:23:15 -0500
Subject: [PATCH 4/7] inclusion feature (#49)
---
CHANGELOG.md | 7 +
README.md | 33 ++-
privaxy/src/resources/proxy.pac.tera | 18 +-
privaxy/src/server/configuration/mod.rs | 48 ++++-
privaxy/src/server/lib.rs | 11 +-
privaxy/src/server/proxy/exclusions.rs | 209 +++++++++++++-----
privaxy/src/server/proxy/mitm.rs | 29 +--
privaxy/src/server/proxy/serve.rs | 43 ++++
privaxy/src/server/web_gui/inclusions.rs | 73 +++++++
privaxy/src/server/web_gui/mod.rs | 10 +
privaxy/src/server/web_gui/pac.rs | 20 ++
privaxy/src/server/web_gui/tls_failures.rs | 4 +-
web_frontend/src/exclusions_page.rs | 9 +-
web_frontend/src/inclusions.rs | 235 +++++++++++++++++++++
web_frontend/src/main.rs | 1 +
web_frontend/src/pac.rs | 3 +
web_frontend/src/settings.rs | 4 +-
17 files changed, 671 insertions(+), 86 deletions(-)
create mode 100644 privaxy/src/server/web_gui/inclusions.rs
create mode 100644 web_frontend/src/inclusions.rs
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 11d45319..1ccc1dc7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,13 @@
## Unreleased
+- Add inclusion-only filtering under Settings → Inclusions & exclusions, with
+ a separate hostname/wildcard list. Exclusions retain precedence; an empty
+ inclusion list bypasses all hosts. The mode applies to HTTP, HTTPS, and PAC
+ generation, updates live through authenticated `GET`/`PUT /api/inclusions`,
+ and reloads from configuration on SIGHUP. Existing configurations keep their
+ previous default mode. Ordinary excluded HTTP requests now also bypass
+ filtering and HTML rewriting.
- Add a Refresh enabled lists action to the Filters page, with progress and
download results, backed by authenticated `POST /api/filters/refresh`.
- Bound Docker log growth in the Compose configuration and documented Docker
diff --git a/README.md b/README.md
index 42ed7b50..e281844c 100644
--- a/README.md
+++ b/README.md
@@ -31,6 +31,7 @@ features, dependency updates, an improved UI, and server-friendly configuration
- [Install the root CA on your client devices](#2-install-the-root-ca-on-your-client-devices)
- [Point clients at the proxy](#3-point-clients-at-the-proxy)
- [Cert-pinned hosts (exclusions)](#4-cert-pinned-hosts-exclusions)
+ - [Filter only selected hosts (inclusions)](#5-filter-only-selected-hosts-inclusions)
- [Screenshots](#screenshots)
- [Acknowledgements](#acknowledgements)
@@ -230,17 +231,45 @@ intercepted. Privaxy handles this two ways:
and not user-editable.
- A **list** of commonly cert-pinned hosts that is
pre-populated into your editable exclusions on first config creation.
- Settings → Exclusions shows the list; the **Reset to defaults** button
+ Settings → Inclusions & exclusions shows the list; the **Reset to defaults** button
re-applies the default list. [Source](privaxy/src/server/proxy/exclusions.rs)
Excluded hosts are still CONNECT-tunneled through the proxy, they're just
not decrypted. Filter rules **do not** apply to their traffic.
If you find a site that breaks under MITM, add its hostname (wildcards
-like `*.example.com` are supported) to Settings → Exclusions and click
+like `*.example.com` are supported) to Settings → Inclusions & exclusions and click
Save. If it's something common, please open an issue so it can be added
to the recommended list.
+### 5. Filter only selected hosts (inclusions)
+
+Under **Settings → Inclusions & exclusions**, enable **Only filter included
+hosts**, enter one hostname or wildcard pattern per line, and save. Use both
+`example.com` and `*.example.com` to cover a domain and its subdomains. Matching
+is case-insensitive and uses the destination hostname without its port.
+
+In this mode, only included hosts are filtered. Everything else passes through
+without ad blocking, HTML/userscript injection, or HTTPS decryption. Explicit
+exclusions and the built-in Apple exclusions always take precedence. An empty
+inclusion list filters nothing. Turning the mode off retains the list and
+restores filtering of all non-excluded hosts.
+
+The generated PAC file uses the same inclusion list and returns `DIRECT` for
+unlisted or excluded hosts; existing PAC bypass rules still take precedence.
+Clients configured with a manual proxy continue forwarding through Privaxy,
+which tunnels unselected HTTPS hosts and forwards unselected HTTP unchanged.
+Clients may need to refresh their PAC file and reopen connections after changes;
+an established HTTPS tunnel keeps its original interception decision.
+
+Inclusions select destination hosts, not entire browsing sessions. Third-party
+ad and resource hosts used by an included site must be included separately if
+you want their requests filtered too.
+
+The equivalent top-level configuration fields are `include_only = true` and
+`inclusions = ["example.com", "*.example.com"]`. Existing configurations default
+to `include_only = false`. File edits take effect on SIGHUP or restart.
+
> **Recovering access**: if you lose the web-UI password, delete the
> `password_hash` value from the config file and restart. The web UI will
> force the setup flow again.
diff --git a/privaxy/src/resources/proxy.pac.tera b/privaxy/src/resources/proxy.pac.tera
index 9454c908..96306fce 100644
--- a/privaxy/src/resources/proxy.pac.tera
+++ b/privaxy/src/resources/proxy.pac.tera
@@ -32,6 +32,22 @@ function FindProxyForURL(url, host) {
if (host === d || dnsDomainIs(host, "." + d)) return "DIRECT";
}
}
+{%- endif %}
+{%- if include_only %}
+ // Exclusions take precedence over inclusions, including built-in bypasses.
+ var exclusions = {{ exclusions | json_encode() | safe }};
+ for (var i = 0; i < exclusions.length; i++) {
+ if (shExpMatch(host, exclusions[i])) return "DIRECT";
+ }
+ var inclusions = {{ inclusions | json_encode() | safe }};
+ var included = false;
+ for (var i = 0; i < inclusions.length; i++) {
+ if (shExpMatch(host, inclusions[i])) {
+ included = true;
+ break;
+ }
+ }
+ if (!included) return "DIRECT";
{%- endif %}
return "PROXY {{ proxy_host }}";
-}
\ No newline at end of file
+}
diff --git a/privaxy/src/server/configuration/mod.rs b/privaxy/src/server/configuration/mod.rs
index 8c8ad108..1cac41ae 100644
--- a/privaxy/src/server/configuration/mod.rs
+++ b/privaxy/src/server/configuration/mod.rs
@@ -66,6 +66,12 @@ pub enum ConfigurationError {
#[derive(Debug, Clone, Deserialize, Serialize, PartialEq, Eq)]
pub struct Configuration {
pub exclusions: BTreeSet,
+ /// Intercept only hosts matching `inclusions`. Existing configurations
+ /// retain the default of intercepting everything except exclusions.
+ #[serde(default)]
+ pub include_only: bool,
+ #[serde(default, skip_serializing_if = "BTreeSet::is_empty")]
+ pub inclusions: BTreeSet,
/// Hosts the operator chose to hide from the TLS-failure report in the
/// web UI. Absent from configuration files written before this feature
/// existed, hence the serde default.
@@ -231,13 +237,13 @@ impl Configuration {
pub async fn set_exclusions(
&mut self,
exclusions: &str,
- mut local_exclusion_store: crate::exclusions::LocalExclusionStore,
+ local_exclusion_store: crate::exclusions::LocalExclusionStore,
) -> ConfigurationResult<()> {
self.exclusions = Self::deserialize_lines(exclusions);
self.save().await?;
- local_exclusion_store.replace_exclusions(Vec::from_iter(self.exclusions.clone()));
+ local_exclusion_store.replace_configuration(self);
Ok(())
}
@@ -576,6 +582,8 @@ impl Configuration {
.iter()
.map(|entry| entry.to_string()),
),
+ include_only: false,
+ inclusions: BTreeSet::new(),
ignored_tls_failures: BTreeSet::new(),
custom_filters: Vec::new(),
auth: Auth::new_initialized(),
@@ -604,6 +612,42 @@ fn get_base_directory() -> ConfigurationResult {
#[cfg(test)]
mod tests {
use super::*;
+ use crate::proxy::exclusions::LocalExclusionStore;
+
+ #[tokio::test]
+ async fn replacing_policy_updates_existing_handles_without_losing_exclusions() {
+ let mut cfg = crate::configuration::Configuration::new_default()
+ .await
+ .unwrap();
+ cfg.exclusions = ["excluded.test".to_string()].into();
+ let store = LocalExclusionStore::from_configuration(&cfg);
+ let existing_handle = store.clone();
+ cfg.include_only = true;
+ cfg.inclusions = ["included.test".to_string(), "excluded.test".to_string()].into();
+ store.replace_configuration(&cfg);
+ assert!(existing_handle.should_intercept("included.test", "included.test"));
+ assert!(!existing_handle.should_intercept("other.test", "other.test"));
+ assert!(!existing_handle.should_intercept("excluded.test", "excluded.test"));
+ }
+
+ #[tokio::test]
+ async fn inclusion_settings_default_off_and_round_trip() {
+ let mut configuration = Configuration::new_default().await.unwrap();
+ let old = toml::to_string_pretty(&configuration)
+ .unwrap()
+ .replace("include_only = false\n", "");
+ let legacy: Configuration = toml::from_str(&old).unwrap();
+ assert!(!legacy.include_only);
+ assert!(legacy.inclusions.is_empty());
+
+ configuration.include_only = true;
+ configuration.inclusions = ["example.com".to_string(), "*.example.com".to_string()].into();
+ let saved = toml::to_string_pretty(&configuration).unwrap();
+ assert_eq!(
+ toml::from_str::(&saved).unwrap(),
+ configuration
+ );
+ }
/// A freshly generated default configuration must survive a
/// `to_string_pretty` -> `from_str` round-trip unchanged. This guards the
diff --git a/privaxy/src/server/lib.rs b/privaxy/src/server/lib.rs
index 280eafb8..7e329554 100644
--- a/privaxy/src/server/lib.rs
+++ b/privaxy/src/server/lib.rs
@@ -197,8 +197,7 @@ pub async fn start_privaxy() -> PrivaxyServer {
// available; the web UI can change it on the fly afterwards.
log_handle.set_level(configuration.debug.log_level.to_level_filter());
- let local_exclusion_store =
- LocalExclusionStore::new(Vec::from_iter(configuration.exclusions.clone()));
+ let local_exclusion_store = LocalExclusionStore::from_configuration(&configuration);
let local_exclusion_store_clone = local_exclusion_store.clone();
let ca_certificate = match configuration.ca.get_ca_certificate().await {
@@ -326,7 +325,6 @@ pub async fn start_privaxy() -> PrivaxyServer {
}
let notify_reload_backend = notify_reload_clone.clone();
let cfg_lock_backend = configuration_save_lock_ref.clone();
- let mut local_exclusion_store = local_exclusion_store;
let mut rt_cert_cache =
cert::CertCache::new(ca_certificate.clone(), ca_private_key.clone());
let mut rt_ca_certificate = ca_certificate;
@@ -360,10 +358,9 @@ pub async fn start_privaxy() -> PrivaxyServer {
continue;
}
};
- // The exclusion store is only otherwise mutated by the web UI
- // route; without this refresh, exclusions edited in the
- // configuration file never take effect on SIGHUP reload.
- local_exclusion_store.replace_exclusions(Vec::from_iter(cfg.exclusions.clone()));
+ // The UI updates this store live; SIGHUP must also pick up file
+ // edits to the mode and both inclusion/exclusion lists.
+ local_exclusion_store.replace_configuration(&cfg);
// Same for the TLS-failure ignore set: pick up hand-edited
// `ignored_tls_failures` entries on SIGHUP reload.
tls_failure_store.set_ignored(cfg.ignored_tls_failures.clone());
diff --git a/privaxy/src/server/proxy/exclusions.rs b/privaxy/src/server/proxy/exclusions.rs
index f6e76717..c6aeb10b 100644
--- a/privaxy/src/server/proxy/exclusions.rs
+++ b/privaxy/src/server/proxy/exclusions.rs
@@ -30,51 +30,51 @@ impl WildMatchCollection {
}
}
-lazy_static! {
- static ref DEFAULT_EXCLUSIONS: WildMatchCollection = {
- // Apple service exclusions, as defined in : https://support.apple.com/en-us/HT210060
- // > Apple services will fail any connection that uses
- // > HTTPS Interception (SSL Inspection). If the HTTPS traffic
- // > traverses a web proxy, disable HTTPS Interception for the hosts
- // > listed in this article.
- let exclusions = vec![
- String::from("*.apple.com"),
- String::from("static.ips.apple.com"),
- String::from("*.push.apple.com"),
- String::from("setup.icloud.com"),
- String::from("*.business.apple.com"),
- String::from("*.school.apple.com"),
- String::from("upload.appleschoolcontent.com"),
- String::from("ws-ee-maidsvc.icloud.com"),
- String::from("itunes.com"),
- String::from("appldnld.apple.com.edgesuite.net"),
- String::from("*.itunes.apple.com"),
- String::from("updates-http.cdn-apple.com"),
- String::from("updates.cdn-apple.com"),
- String::from("*.apps.apple.com"),
- String::from("*.mzstatic.com"),
- String::from("*.appattest.apple.com"),
- String::from("doh.dns.apple.com"),
- String::from("appleid.cdn-apple.com"),
- String::from("*.apple-cloudkit.com"),
- String::from("*.apple-livephotoskit.com"),
- String::from("*.apzones.com"),
- String::from("*.cdn-apple.com"),
- String::from("*.gc.apple.com"),
- String::from("*.icloud.com"),
- String::from("*.icloud.com.cn"),
- String::from("*.icloud.apple.com"),
- String::from("*.icloud-content.com"),
- String::from("*.iwork.apple.com"),
- String::from("mask.icloud.com"),
- String::from("mask-h2.icloud.com"),
- String::from("mask-api.icloud.com"),
- String::from("devimages-cdn.apple.com"),
- String::from("download.developer.apple.com"),
- ];
+// Apple service exclusions, as defined in https://support.apple.com/en-us/HT210060.
+// Shared with PAC generation so inclusion mode keeps the same precedence there.
+pub(crate) const DEFAULT_EXCLUSION_PATTERNS: &[&str] = &[
+ "*.apple.com",
+ "static.ips.apple.com",
+ "*.push.apple.com",
+ "setup.icloud.com",
+ "*.business.apple.com",
+ "*.school.apple.com",
+ "upload.appleschoolcontent.com",
+ "ws-ee-maidsvc.icloud.com",
+ "itunes.com",
+ "appldnld.apple.com.edgesuite.net",
+ "*.itunes.apple.com",
+ "updates-http.cdn-apple.com",
+ "updates.cdn-apple.com",
+ "*.apps.apple.com",
+ "*.mzstatic.com",
+ "*.appattest.apple.com",
+ "doh.dns.apple.com",
+ "appleid.cdn-apple.com",
+ "*.apple-cloudkit.com",
+ "*.apple-livephotoskit.com",
+ "*.apzones.com",
+ "*.cdn-apple.com",
+ "*.gc.apple.com",
+ "*.icloud.com",
+ "*.icloud.com.cn",
+ "*.icloud.apple.com",
+ "*.icloud-content.com",
+ "*.iwork.apple.com",
+ "mask.icloud.com",
+ "mask-h2.icloud.com",
+ "mask-api.icloud.com",
+ "devimages-cdn.apple.com",
+ "download.developer.apple.com",
+];
- WildMatchCollection::new(exclusions)
- };
+lazy_static! {
+ static ref DEFAULT_EXCLUSIONS: WildMatchCollection = WildMatchCollection::new(
+ DEFAULT_EXCLUSION_PATTERNS
+ .iter()
+ .map(|host| host.to_string())
+ .collect()
+ );
}
/// Hosts the maintainer has observed to use certificate pinning, HSTS preload
@@ -272,26 +272,125 @@ pub fn recommended_exclusions() -> &'static [&'static str] {
]
}
+/// One atomic snapshot keeps the mode and both lists consistent for a request.
#[derive(Debug, Clone)]
-pub struct LocalExclusionStore(Arc>);
+struct InterceptionRules {
+ exclusions: WildMatchCollection,
+ inclusions: WildMatchCollection,
+ include_only: bool,
+}
+
+impl From<&crate::configuration::Configuration> for InterceptionRules {
+ fn from(configuration: &crate::configuration::Configuration) -> Self {
+ Self {
+ exclusions: WildMatchCollection::new(
+ configuration.exclusions.iter().cloned().collect(),
+ ),
+ inclusions: WildMatchCollection::new(
+ configuration.inclusions.iter().cloned().collect(),
+ ),
+ include_only: configuration.include_only,
+ }
+ }
+}
+
+#[derive(Debug, Clone)]
+pub struct LocalExclusionStore(Arc>);
impl LocalExclusionStore {
+ #[cfg(test)]
pub fn new(exclusions: Vec) -> Self {
- let collection = WildMatchCollection::new(exclusions);
- Self(Arc::new(RwLock::new(collection)))
+ Self(Arc::new(RwLock::new(InterceptionRules {
+ exclusions: WildMatchCollection::new(exclusions),
+ inclusions: WildMatchCollection::new(Vec::new()),
+ include_only: false,
+ })))
}
- pub fn replace_exclusions(&mut self, exclusions: Vec) {
- let new_exclusion_store = LocalExclusionStore::new(exclusions);
+ pub fn from_configuration(configuration: &crate::configuration::Configuration) -> Self {
+ Self(Arc::new(RwLock::new(configuration.into())))
+ }
+
+ pub fn replace_configuration(&self, configuration: &crate::configuration::Configuration) {
+ *self.0.write().unwrap() = configuration.into();
+ }
+
+ /// Match both forms of a CONNECT hostname (e.g. rbm.goog(smsft) and
+ /// rbm.goog) before applying the mode. Negating each match independently
+ /// would incorrectly bypass a host whose inclusion matches just one form.
+ pub fn should_intercept(&self, host: &str, raw_host: &str) -> bool {
+ let rules = self.0.read().unwrap();
+ let matches = |list: &WildMatchCollection| {
+ list.is_match(host) || (raw_host != host && list.is_match(raw_host))
+ };
+ !matches(&DEFAULT_EXCLUSIONS)
+ && !matches(&rules.exclusions)
+ && (!rules.include_only || matches(&rules.inclusions))
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ fn policy(include_only: bool, inclusions: &[&str], exclusions: &[&str]) -> LocalExclusionStore {
+ LocalExclusionStore(Arc::new(RwLock::new(InterceptionRules {
+ include_only,
+ inclusions: WildMatchCollection::new(
+ inclusions.iter().map(|s| s.to_string()).collect(),
+ ),
+ exclusions: WildMatchCollection::new(
+ exclusions.iter().map(|s| s.to_string()).collect(),
+ ),
+ })))
+ }
+
+ #[test]
+ fn normal_mode_preserves_exclusions_and_ignores_inclusions() {
+ let store = policy(false, &["included.test"], &["excluded.test"]);
+ assert!(store.should_intercept("other.test", "other.test"));
+ assert!(!store.should_intercept("excluded.test", "excluded.test"));
+ assert!(!store.should_intercept("setup.icloud.com", "setup.icloud.com"));
+ }
+
+ #[test]
+ fn inclusion_mode_matches_exact_and_wildcard_hosts_case_insensitively() {
+ let store = policy(
+ true,
+ &["EXAMPLE.com", "*.example.net", "192.0.2.*", "[::1]"],
+ &[],
+ );
+ for host in ["example.com", "Sub.Example.NET", "192.0.2.42", "[::1]"] {
+ assert!(store.should_intercept(host, host), "{host}");
+ }
+ for host in [
+ "sub.example.com",
+ "example.net",
+ "example.com.evil.test",
+ "other.test",
+ ] {
+ assert!(!store.should_intercept(host, host), "{host}");
+ }
+ assert!(!policy(true, &[], &[]).should_intercept("example.com", "example.com"));
+ }
- *self.0.write().unwrap() = new_exclusion_store.0.read().unwrap().clone();
+ #[test]
+ fn exclusions_always_win_even_when_everything_is_included() {
+ let store = policy(true, &["*"], &["*.example.com"]);
+ assert!(store.should_intercept("other.test", "other.test"));
+ assert!(!store.should_intercept("sub.example.com", "sub.example.com"));
+ assert!(!store.should_intercept("setup.icloud.com", "setup.icloud.com"));
}
- pub fn contains(&self, element: &str) -> bool {
- if DEFAULT_EXCLUSIONS.is_match(element) {
- true
- } else {
- self.0.read().unwrap().is_match(element)
+ #[test]
+ fn service_selector_aliases_are_matched_before_applying_the_mode() {
+ for included in ["rbm.goog", "rbm.goog(smsft)"] {
+ let store = policy(true, &[included], &[]);
+ assert!(store.should_intercept("rbm.goog", "rbm.goog(smsft)"));
+ for excluded in ["rbm.goog", "rbm.goog(smsft)"] {
+ assert!(!policy(true, &[included], &[excluded])
+ .should_intercept("rbm.goog", "rbm.goog(smsft)"));
+ }
}
}
}
diff --git a/privaxy/src/server/proxy/mitm.rs b/privaxy/src/server/proxy/mitm.rs
index 7967c2be..28e1514b 100644
--- a/privaxy/src/server/proxy/mitm.rs
+++ b/privaxy/src/server/proxy/mitm.rs
@@ -73,7 +73,7 @@ pub(crate) async fn serve_mitm_session(
// cert, so minting one would only delay the CONNECT response (cert
// signing is expensive on low-powered machines).
let server_configuration =
- if is_authority_excluded(&local_exclusion_store, &authority, &raw_authority) {
+ if !should_intercept_authority(&local_exclusion_store, &authority, &raw_authority) {
None
} else {
Some(Arc::new(
@@ -128,6 +128,11 @@ pub(crate) async fn serve_mitm_session(
scriptlet_debug_logging,
gui_base_url.clone(),
user_scripts.clone(),
+ should_intercept_authority(
+ &local_exclusion_store,
+ &authority,
+ &raw_authority,
+ ),
)
}),
)
@@ -169,7 +174,7 @@ pub(crate) async fn serve_mitm_session(
});
Ok(Response::new(empty_body()))
- } else if is_authority_excluded(&local_exclusion_store, &authority, &raw_authority)
+ } else if !should_intercept_authority(&local_exclusion_store, &authority, &raw_authority)
&& is_opaque_upgrade(req.headers())
{
// An excluded host performing a protocol upgrade over plain HTTP — e.g.
@@ -199,6 +204,8 @@ pub(crate) async fn serve_mitm_session(
);
}
+ let intercept =
+ should_intercept_authority(&local_exclusion_store, &authority, &raw_authority);
serve(
adblock_requester,
req,
@@ -213,6 +220,7 @@ pub(crate) async fn serve_mitm_session(
scriptlet_debug_logging,
gui_base_url,
user_scripts,
+ intercept,
)
.await
}
@@ -431,16 +439,13 @@ fn sanitize_authority(authority: &Authority) -> Authority {
candidate.parse().unwrap_or_else(|_| authority.clone())
}
-/// An authority is excluded when its sanitized host matches the exclusion
-/// list, or — for authorities that carried a service selector — when the raw
-/// client-sent host matches a literal exclusion entry.
-fn is_authority_excluded(
+/// Apply one interception decision to the sanitized and original hostnames.
+fn should_intercept_authority(
exclusions: &LocalExclusionStore,
authority: &Authority,
raw_authority: &Authority,
) -> bool {
- exclusions.contains(authority.host())
- || (raw_authority.host() != authority.host() && exclusions.contains(raw_authority.host()))
+ exclusions.should_intercept(authority.host(), raw_authority.host())
}
/// Pipe two duplex streams in both directions until either side closes.
@@ -542,16 +547,16 @@ mod tests {
let sanitized = sanitize_authority(&raw);
let exact = LocalExclusionStore::new(vec![String::from("rbm.goog")]);
- assert!(is_authority_excluded(&exact, &sanitized, &raw));
+ assert!(!should_intercept_authority(&exact, &sanitized, &raw));
let wildcard = LocalExclusionStore::new(vec![String::from("*.goog")]);
- assert!(is_authority_excluded(&wildcard, &sanitized, &raw));
+ assert!(!should_intercept_authority(&wildcard, &sanitized, &raw));
// An entry matching the literal client-sent host keeps working.
let literal = LocalExclusionStore::new(vec![String::from("rbm.goog(smsft)")]);
- assert!(is_authority_excluded(&literal, &sanitized, &raw));
+ assert!(!should_intercept_authority(&literal, &sanitized, &raw));
let unrelated = LocalExclusionStore::new(vec![String::from("example.com")]);
- assert!(!is_authority_excluded(&unrelated, &sanitized, &raw));
+ assert!(should_intercept_authority(&unrelated, &sanitized, &raw));
}
}
diff --git a/privaxy/src/server/proxy/serve.rs b/privaxy/src/server/proxy/serve.rs
index 98344122..17e481ce 100644
--- a/privaxy/src/server/proxy/serve.rs
+++ b/privaxy/src/server/proxy/serve.rs
@@ -300,6 +300,7 @@ pub(crate) async fn serve(
scriptlet_debug_logging: bool,
gui_base_url: Option,
user_scripts: UserScriptContext,
+ intercept: bool,
) -> Result, hyper::Error> {
let scheme_string = scheme.to_string();
@@ -323,6 +324,11 @@ pub(crate) async fn serve(
// running in the page's main world reach Privaxy without CORS. Handled
// before any statistics counting: these are not proxied traffic.
if gm_endpoint::is_reserved(uri.path()) {
+ // A page opened before a policy change may still call its injected
+ // endpoints. Revoke them without leaking tokens/storage upstream.
+ if !intercept {
+ return Ok(get_empty_response(StatusCode::NOT_FOUND));
+ }
let (parts, body) = request.into_parts();
// Bounded so a hostile page cannot make the proxy buffer an unbounded
// body; an oversized request simply fails to parse below.
@@ -346,6 +352,43 @@ pub(crate) async fn serve(
statistics.increment_top_clients(client_ip_address);
+ if !intercept {
+ let mut req = req;
+ req.headers_mut().remove(http::header::PROXY_AUTHORIZATION);
+ req.headers_mut().remove("proxy-connection");
+ if broadcast_sender.receiver_count() > 0 {
+ let _ = broadcast_sender.send(Event {
+ now: chrono::Utc::now(),
+ method: req.method().to_string(),
+ url: uri.to_string(),
+ is_request_blocked: false,
+ });
+ }
+ // Forward with hyper so bodies, compression and CSP remain untouched.
+ // This path skips adblock, DoH policy and all HTML/userscript injection.
+ let response = if req.headers().contains_key(http::header::UPGRADE) {
+ perform_two_ends_upgrade(req, uri, hyper_client).await
+ } else {
+ let (mut parts, body) = req.into_parts();
+ parts.headers.remove(http::header::HOST);
+ // This client speaks HTTP/1 upstream, including for a request
+ // arriving over an existing intercepted HTTP/2 connection.
+ parts.version = http::Version::HTTP_11;
+ match hyper_client
+ .request(Request::from_parts(parts, boxed_incoming(body)))
+ .await
+ {
+ Ok(response) => response.map(boxed_incoming),
+ Err(error) => {
+ log::warn!("Unable to forward unfiltered request to {uri}: {error}");
+ return Ok(get_empty_response(StatusCode::BAD_GATEWAY));
+ }
+ }
+ };
+ statistics.increment_proxied_requests();
+ return Ok(response);
+ }
+
let request_type = request_type_from_headers(req.headers());
// Canonical URL (default port stripped) for all adblock-engine matching —
diff --git a/privaxy/src/server/web_gui/inclusions.rs b/privaxy/src/server/web_gui/inclusions.rs
new file mode 100644
index 00000000..2f221438
--- /dev/null
+++ b/privaxy/src/server/web_gui/inclusions.rs
@@ -0,0 +1,73 @@
+use super::{get_error_response, get_unprocessable_response};
+use crate::{configuration::Configuration, proxy::exclusions::LocalExclusionStore};
+use serde::{Deserialize, Serialize};
+use std::{convert::Infallible, sync::Arc};
+use tokio::sync::Mutex;
+use warp::{filters::BoxedFilter, http::StatusCode, Filter};
+
+#[derive(Deserialize, Serialize)]
+struct InclusionSettings {
+ include_only: bool,
+ inclusions: Vec,
+}
+
+async fn get_inclusions() -> Result, Infallible> {
+ match Configuration::read_from_home().await {
+ Ok(configuration) => Ok(Box::new(warp::reply::json(&InclusionSettings {
+ include_only: configuration.include_only,
+ inclusions: configuration.inclusions.into_iter().collect(),
+ }))),
+ Err(error) => Ok(Box::new(get_error_response(error))),
+ }
+}
+
+async fn put_inclusions(
+ settings: InclusionSettings,
+ save_lock: Arc>,
+ store: LocalExclusionStore,
+) -> Result, Infallible> {
+ let inclusions: Vec = settings
+ .inclusions
+ .iter()
+ .map(|host| host.trim().to_lowercase())
+ .filter(|host| !host.is_empty())
+ .collect();
+ if inclusions
+ .iter()
+ .any(|host| host.chars().any(char::is_whitespace) || host.contains('/'))
+ {
+ return Ok(Box::new(get_unprocessable_response(
+ "Use one hostname or wildcard pattern per entry, without a URL or path",
+ )));
+ }
+
+ let _guard = save_lock.lock().await;
+ let mut configuration = match Configuration::read_from_home().await {
+ Ok(configuration) => configuration,
+ Err(error) => return Ok(Box::new(get_error_response(error))),
+ };
+ configuration.include_only = settings.include_only;
+ configuration.inclusions = inclusions.into_iter().collect();
+ if let Err(error) = configuration.save().await {
+ return Ok(Box::new(get_error_response(error)));
+ }
+ store.replace_configuration(&configuration);
+ // No filter-engine rebuild or listener restart is needed. PAC requests
+ // read the persisted configuration, and new proxy requests use this store.
+ Ok(Box::new(StatusCode::ACCEPTED))
+}
+
+pub(super) fn create_routes(
+ save_lock: Arc>,
+ store: LocalExclusionStore,
+) -> BoxedFilter<(impl warp::Reply,)> {
+ let get = warp::path::end().and(warp::get()).and_then(get_inclusions);
+ let put = warp::path::end()
+ .and(warp::put())
+ .and(warp::body::content_length_limit(1024 * 1024))
+ .and(warp::body::json())
+ .and(super::with_configuration_save_lock(save_lock))
+ .and(super::with_local_exclusions_store(store))
+ .and_then(put_inclusions);
+ get.or(put).boxed()
+}
diff --git a/privaxy/src/server/web_gui/mod.rs b/privaxy/src/server/web_gui/mod.rs
index 0c865253..54209ce0 100644
--- a/privaxy/src/server/web_gui/mod.rs
+++ b/privaxy/src/server/web_gui/mod.rs
@@ -23,6 +23,7 @@ pub(crate) mod events;
pub(crate) mod exclusions;
mod filterlists;
pub(crate) mod filters;
+mod inclusions;
pub(crate) mod logs;
mod pac;
pub(crate) mod settings;
@@ -198,6 +199,14 @@ fn create_api_routes(
local_exclusions_store.clone(),
));
+ let inclusions_route =
+ warp::path("inclusions")
+ .and(require_auth.clone())
+ .and(inclusions::create_routes(
+ configuration_save_lock.clone(),
+ local_exclusions_store.clone(),
+ ));
+
let settings_route =
warp::path("settings")
.and(require_auth.clone())
@@ -252,6 +261,7 @@ fn create_api_routes(
.or(filters_route)
.or(custom_filters_route)
.or(exclusions_route)
+ .or(inclusions_route)
.or(blocking_enabled_route)
.or(tls_failures_route)
.or(settings_route)
diff --git a/privaxy/src/server/web_gui/pac.rs b/privaxy/src/server/web_gui/pac.rs
index a7f5cd57..286034d9 100644
--- a/privaxy/src/server/web_gui/pac.rs
+++ b/privaxy/src/server/web_gui/pac.rs
@@ -1,4 +1,5 @@
use crate::configuration::Configuration;
+use crate::proxy::exclusions::DEFAULT_EXCLUSION_PATTERNS;
use serde::Serialize;
use std::collections::BTreeMap;
use std::sync::Arc;
@@ -17,6 +18,9 @@ struct PacContext<'a> {
ips: &'a [String],
cidrs: &'a BTreeMap,
fqdns: &'a [String],
+ include_only: bool,
+ inclusions: Vec,
+ exclusions: Vec,
}
pub(crate) fn create_routes(configuration_save_lock: Arc>) -> BoxedFilter<(impl Reply,)> {
@@ -69,6 +73,22 @@ async fn render_pac(
ips: &cfg.network.pac_direct_ips,
cidrs: &cidrs,
fqdns: &cfg.network.pac_direct_fqdns,
+ include_only: cfg.include_only,
+ inclusions: cfg
+ .inclusions
+ .iter()
+ .map(|host| host.to_lowercase())
+ .collect(),
+ exclusions: cfg
+ .exclusions
+ .iter()
+ .map(|host| host.to_lowercase())
+ .chain(
+ DEFAULT_EXCLUSION_PATTERNS
+ .iter()
+ .map(|host| host.to_string()),
+ )
+ .collect(),
};
let ctx = Context::from_serialize(&ctx_data).map_err(|err| {
diff --git a/privaxy/src/server/web_gui/tls_failures.rs b/privaxy/src/server/web_gui/tls_failures.rs
index e7f4f077..d6d0167d 100644
--- a/privaxy/src/server/web_gui/tls_failures.rs
+++ b/privaxy/src/server/web_gui/tls_failures.rs
@@ -19,12 +19,12 @@ async fn get_tls_failures(
local_exclusions_store: LocalExclusionStore,
) -> Result, Infallible> {
// The store already omits ignored hosts; additionally omit hosts the
- // exclusion list now covers (their failures predate the exclusion, or the
+ // interception policy now bypasses (their failures predate the change, or the
// client re-dialed before the exclusion took effect).
let entries: Vec = tls_failure_store
.entries()
.into_iter()
- .filter(|entry| !local_exclusions_store.contains(&entry.host))
+ .filter(|entry| local_exclusions_store.should_intercept(&entry.host, &entry.host))
.collect();
Ok(Box::new(warp::reply::json(&entries)))
diff --git a/web_frontend/src/exclusions_page.rs b/web_frontend/src/exclusions_page.rs
index 90bd8304..813d287d 100644
--- a/web_frontend/src/exclusions_page.rs
+++ b/web_frontend/src/exclusions_page.rs
@@ -1,3 +1,4 @@
+use crate::inclusions::Inclusions;
use crate::settings_textarea::SettingsTextarea;
use crate::tls_failures::TlsFailuresPanel;
use yew::{html, Component, Context, Html};
@@ -41,8 +42,7 @@ impl Component for ExclusionsPage {
let description = html! {
- {"Exclusions are hosts or domains that are not passed through the MITM pipeline. "}
- {"Excluded entries will be transparently tunneled."}
+ {"Excluded hosts pass through without filtering or HTTPS interception. "}
{"Use "}{"Reset to defaults"}
@@ -55,7 +55,10 @@ impl Component for ExclusionsPage {
html! {
<>
-
+
+