From 903a9d04694850e7ca9450897d838cb2ded38b25 Mon Sep 17 00:00:00 2001 From: Christophe Pettus Date: Tue, 21 Jul 2026 12:20:11 -0700 Subject: [PATCH 1/2] Add design spec for transaction_cost_limit (issue #4) --- ...026-07-21-transaction-cost-limit-design.md | 85 +++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md diff --git a/docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md b/docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md new file mode 100644 index 0000000..5e64d49 --- /dev/null +++ b/docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md @@ -0,0 +1,85 @@ +# Design: plan_filter.transaction_cost_limit + +Implements [issue #4](https://github.com/pgexperts/pg_plan_filter/issues/4): +cap the cumulative estimated plan cost of the statements executed within a +single transaction, so a batch interface cannot smuggle unbounded work +through many individually-cheap statements. Requested by PostgREST for +mapping to HTTP 429 at the application edge. + +## GUC + +`plan_filter.transaction_cost_limit` — real, default 0 (feature off), +range 0..DBL_MAX, `PGC_SUSET`, covered by the reserved `plan_filter.` +prefix. Errors use the same SQLSTATE as the statement limit +(`ERRCODE_STATEMENT_TOO_COMPLEX`, 54001) so one client-side mapping +catches both, with a distinct message: `transaction cost limit exceeded`. + +## Accounting model (the load-bearing decision) + +Cost is accumulated **per execution, not per plan**, in a new +`ExecutorStart` hook (the existing planner-hook statement check is +untouched). Rationale: repeated `EXECUTE` of a prepared statement stops +invoking the planner once a generic plan is cached, so plan-time +accounting can be defeated by a `PREPARE` + `EXECUTE` loop — exactly the +misbehaving-batch-client threat the feature exists for. Every execution +passes through `ExecutorStart`. + +The hook, before chaining to `standard_ExecutorStart` (so a doomed +statement never begins executing): + +- skips `EXEC_FLAG_EXPLAIN_ONLY` (plain `EXPLAIN` neither consumes budget + nor is blocked by an exhausted one — it remains the documented way to + inspect an expensive plan); +- skips `IsParallelWorker()` (leader already counted the plan); +- skips non-SELECT when `filter_select_only` is on (mirrors the statement + check); +- otherwise **always adds** `plannedstmt->planTree->total_cost` to the + accumulator, and raises the error only when the limit is non-zero and + exceeded. Always-accumulate means `SET LOCAL` can disable the check + but cannot erase history. + +Utility statements never reach the executor and are not counted — the +same statements the per-statement limit has never seen. + +## State and reset + +One `static double xact_cost_total`, zeroed in a `RegisterXactCallback` +callback on `XACT_EVENT_COMMIT`, `XACT_EVENT_PARALLEL_COMMIT`, +`XACT_EVENT_ABORT`, `XACT_EVENT_PARALLEL_ABORT`, and +`XACT_EVENT_PREPARE` (the callback only assigns a double — safe in that +context). No subtransaction callback: `ROLLBACK TO SAVEPOINT` does +**not** refund cost. The limit is a resource-consumption guard; the +work was attempted, and refusing refunds also removes the need to track +a savepoint stack. + +## Tests (extend the existing suite; same one-expected-file discipline) + +1. Two ~190-cost SELECTs under limit 300 in one transaction: first + passes, second errors. +2. Reset: a fresh transaction passes again after COMMIT and after + ROLLBACK of an aborted transaction. +3. No refund: pass, SAVEPOINT, error, `ROLLBACK TO`, next statement + still errors. +4. Execution accounting discriminator: `plan_cache_mode = + force_generic_plan`, `PREPARE`, two `EXECUTE`s — the second must + error (it would not under plan-time accounting). +5. Plain `EXPLAIN` consumes nothing: after one SELECT (~190/300), two + consecutive `EXPLAIN (COSTS OFF)` both succeed (if EXPLAIN consumed, + the second would error). +6. SUSET: unprivileged `SET plan_filter.transaction_cost_limit` is + denied. + +The suite's post-`LOAD` pin block also pins the new GUC to 0. + +## Documentation + +README section for the new GUC (semantics above, spelled out); ADR 0004 +recording execution-time accounting with plan-time as the rejected +alternative; ARCHITECTURE.md map/invariants updated. + +## Compatibility + +`ExecutorStart_hook_type (QueryDesc *, int)`, `RegisterXactCallback`, +`IsParallelWorker()`, and `plan_cache_mode` are uniform across +PostgreSQL 14-18 (PG 18's bool-returning `ExecutorStart` experiment was +reverted before release). The CI matrix proves each major. From 5275cd2f70e2e14d5f68e9fb629e5cf82b224b2a Mon Sep 17 00:00:00 2001 From: Christophe Pettus Date: Tue, 21 Jul 2026 12:46:10 -0700 Subject: [PATCH 2/2] Add plan_filter.transaction_cost_limit (issue #4) Caps the cumulative estimated plan cost of the statements executed in one transaction, so a batch interface cannot smuggle unbounded work through many individually-cheap statements. Requested for throttling PostgREST batch endpoints and mapping the error to HTTP 429. - New PGC_SUSET GUC plan_filter.transaction_cost_limit, default 0 (off), same SQLSTATE (54001) as the statement limit - Accounting happens in a new ExecutorStart hook, not the planner hook, so every execution is charged: a PREPARE/EXECUTE loop over a cached generic plan cannot evade a plan-time accumulator - A transaction callback zeroes the accumulator on commit/abort/prepare; ROLLBACK TO SAVEPOINT does not refund cost already charged - Plain EXPLAIN and parallel workers are exempt; filter_select_only applies as it does to the statement limit - Regression tests cover the reset, no-refund, generic-plan, EXPLAIN, filter_select_only, and SQLSTATE behaviors - README, ADR 0004, and ARCHITECTURE.md document the design, and warn that plan-cost limits are not a security boundary against a hostile role: the cost estimate is computed from USERSET planner GUCs a client can lower to evade either limit --- ARCHITECTURE.md | 35 +++- README.md | 41 ++++- ...saction-cost-limit-execution-accounting.md | 79 ++++++++ docs/adr/README.md | 1 + ...026-07-21-transaction-cost-limit-design.md | 18 +- expected/plan_filter.out | 171 +++++++++++++++++- plan_filter.c | 105 ++++++++++- sql/plan_filter.sql | 109 ++++++++++- 8 files changed, 542 insertions(+), 17 deletions(-) create mode 100644 docs/adr/0004-transaction-cost-limit-execution-accounting.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d8715c2..aaeca77 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,10 +1,13 @@ # ARCHITECTURE `pg_plan_filter` is a single-file PostgreSQL loadable module (not an -extension: no `.control`, no SQL objects) that installs a `planner_hook` -and raises `ERRCODE_STATEMENT_TOO_COMPLEX` when a completed plan's -`total_cost` exceeds `plan_filter.statement_cost_limit`. Decision -rationale lives in `docs/adr/`; this file is the map and the invariants. +extension: no `.control`, no SQL objects) that raises +`ERRCODE_STATEMENT_TOO_COMPLEX` when estimated plan cost exceeds a +configured limit: a `planner_hook` enforces the per-statement limit +(`plan_filter.statement_cost_limit`), and an `ExecutorStart` hook plus a +transaction callback enforce the per-transaction limit +(`plan_filter.transaction_cost_limit`). Decision rationale lives in +`docs/adr/`; this file is the map and the invariants. ## Map @@ -27,8 +30,18 @@ rationale lives in `docs/adr/`; this file is the map and the invariants. - **Hook discipline**: `limit_func` calls exactly one of `prev_planner_hook` or `standard_planner`, and checks the cost only *after* planning completes, on the finished `PlannedStmt`. The error - is thrown, never returned. -- **Both feature GUCs are `PGC_SUSET` on purpose** — an unprivileged user + is thrown, never returned. `xact_limit_func` charges and checks + *before* chaining to `standard_ExecutorStart`, so an over-budget + statement never begins executing. +- **Transaction accounting is per execution, not per plan** (ADR 0004): + it lives in `ExecutorStart` so cached generic plans from repeated + `EXECUTE` are charged every time. The accumulator always accrues + (limit unset only disables the check), is zeroed only by the + end-of-transaction callback, and is never refunded by + `ROLLBACK TO SAVEPOINT`. `EXEC_FLAG_EXPLAIN_ONLY` and + `IsParallelWorker()` executions are exempt — keep them so + (double-charging and EXPLAIN-blocking are the failure modes). +- **All feature GUCs are `PGC_SUSET` on purpose** — an unprivileged user must not be able to lift an administrator's limit. The regression suite tests this; do not downgrade the context. - **`plan_filter.module_loaded`** exists only as a presence signal @@ -56,6 +69,16 @@ rationale lives in `docs/adr/`; this file is the map and the invariants. - `filter_select_only` exempts by `parse->commandType != CMD_SELECT`; a SELECT with data-modifying CTEs is still `CMD_SELECT` and still filtered — SELECT ≠ read-only, as the README warns. +- **Neither limit is a security boundary against a hostile role.** Both + compare against the planner's estimate, which comes from `USERSET` + cost GUCs (`seq_page_cost` et al.) any `SET`-capable role can zero. + `PGC_SUSET` on the limit GUCs stops a user *raising the limit* — not + the estimate bypass. Do not let the SUSET invariant above read as + adversary-proof (ADR 0004). It guards careless load, not attackers. +- Regression tests that depend on a cost *ratio* (the transaction-limit + section) use their own freshly-created table: earlier tests leave dead + tuples in `plan_filter_test`, which roughly doubles its seq-scan cost + and silently shifts thresholds. ## Working on this repo diff --git a/README.md b/README.md index 4872052..03d0e4e 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,30 @@ which means do not apply any filter. So a typical pair of settings in the turns it on. Be aware that SELECT != READONLY, since SELECT statements might also modify data. +`transaction_cost_limit` limits the *combined* estimated cost of the +statements executed within a single transaction, raising the same error +class once the running total would be exceeded. This is aimed at batch +interfaces: many individually-cheap statements can no longer add up to +unbounded work. The default of 0 turns it off. + + plan_filter.transaction_cost_limit = 500000.0 + +Details of the accounting: + +- Cost is charged per *execution*, so repeatedly executing a prepared + statement counts every execution, even after PostgreSQL switches to a + cached generic plan. +- Plain `EXPLAIN` neither consumes budget nor is refused once the budget + is exhausted, so plans remain inspectable. +- `ROLLBACK TO SAVEPOINT` does not refund cost already charged: the limit + guards attempted resource consumption, not net effect. +- Statements that execute without a plan — most DDL, such as `CREATE + INDEX` — are not counted. Utility statements that *wrap* a query, + however, do execute a plan and are counted: `CREATE TABLE ... AS`, + `CREATE MATERIALIZED VIEW`, `COPY (SELECT ...) TO`. This mirrors + `statement_cost_limit`, which sees the same statements. +- When `filter_select_only` is on, it applies to this limit as well. + When this module is running with a non-zero `statement_cost_limit`, it will also prevent `EXPLAIN` on expensive queries. The solution would be to `set statement_cost_limit` temporarily to 0 and then run the `EXPLAIN`, @@ -79,12 +103,25 @@ so nothing needs to be added to `shared_preload_libraries` first. Warnings -------- -`statement_cost_limit` will cancel plans based on their estimated cost. The PostgreSQL +Both limits cancel plans based on their estimated cost. The PostgreSQL planner can and does return cost estimates which are unrelated to the actual query execution time. As such, you should be prepared for "false positive" -cancellations if you use `pg_plan_filter`, and you should set `statement_cost_limit` +cancellations if you use `pg_plan_filter`, and you should set the limits generously. +More importantly, the estimate these limits compare against is computed from +the planner cost settings — `seq_page_cost`, `cpu_tuple_cost`, the +parallelism costs, and so on — and every one of those is `USERSET`: any role +that can run `SET` can lower them, even to zero, and so deflate the estimated +cost of its own statements below any limit. The limit GUCs themselves are +superuser-only and cannot be raised by an ordinary user, but that does not +close this gap, because the attacker changes the number being compared rather +than the limit. Treat `pg_plan_filter` as a cooperative guard against +accidental or careless load — a misconfigured ORM, a runaway report — not as +a hard security boundary against a hostile role that controls its own +session. For the batch-throttling use case in particular, keep it as one +layer behind an application-level limit rather than the sole defense. + Credits ------- diff --git a/docs/adr/0004-transaction-cost-limit-execution-accounting.md b/docs/adr/0004-transaction-cost-limit-execution-accounting.md new file mode 100644 index 0000000..aeaf847 --- /dev/null +++ b/docs/adr/0004-transaction-cost-limit-execution-accounting.md @@ -0,0 +1,79 @@ +--- +id: 0004 +title: Transaction cost limit accumulates at execution time, without savepoint refunds +date: 2026-07-21 +status: Accepted +summary: transaction_cost_limit charges each execution's plan cost in an ExecutorStart hook, resets via xact callback, and never refunds rolled-back subtransactions. +--- + +# 0004. Transaction cost limit accumulates at execution time, without savepoint refunds + +## Context + +Issue #4 (from PostgREST) asks for `plan_filter.transaction_cost_limit`: +a cap on the *sum* of statement costs within one transaction, so a batch +endpoint cannot smuggle unbounded work through many individually-cheap +statements, with the error mapped to HTTP 429 at the application edge. +The module was purely stateless per statement; this feature needs +cross-statement state and a decision about *when* cost is charged. + +## Decision + +Charge cost per **execution** in a new `ExecutorStart` hook: add the +finished plan's `total_cost` to a static per-backend accumulator, error +(same SQLSTATE 54001 as the statement limit, distinct message) when a +non-zero limit is exceeded, before the statement begins executing. The +accumulator always accrues, even with the limit unset, so `SET LOCAL` +can disable the check but not erase history. It is zeroed by a +`RegisterXactCallback` callback on commit, abort, and prepare (including +the parallel variants). Skipped entirely: `EXEC_FLAG_EXPLAIN_ONLY` +(plain `EXPLAIN` stays free and usable), parallel workers (the leader +already charged the plan), and non-SELECT when `filter_select_only` is +on (mirrors the statement check). `ROLLBACK TO SAVEPOINT` does not +refund cost. The existing per-statement planner-hook check is +untouched. + +## Alternatives considered + +- **Accumulate in the existing planner hook** — smallest diff, but + defeated by the exact threat the feature targets: repeated `EXECUTE` + of a prepared statement stops invoking the planner once a generic + plan is cached, so a PREPARE/EXECUTE loop escapes a plan-time + accumulator. It also charges plain `EXPLAIN`. The regression suite + contains a `force_generic_plan` test that fails under this design. +- **Move the per-statement check to ExecutorStart too** — one uniform + accounting point, but a silent behavior change to the existing + feature (plain `EXPLAIN` of an over-limit query is deliberately + blocked by the statement filter today, and its documented escape + hatch depends on that). +- **Refund savepoint rollbacks** — more "correct" as bookkeeping, but + the limit guards attempted resource consumption (the work was done), + and refunds would require tracking a subtransaction cost stack for a + case the security use case does not want refunded anyway. + +## Consequences + +*Careless* batch clients are bounded even through prepared-statement +loops. A *hostile* client is not: the charged value is the planner's +estimate, computed from cost GUCs (`seq_page_cost`, `cpu_tuple_cost`, +the parallelism costs) that are all `USERSET`, so any role that can run +`SET` can drive its own estimated costs to zero and evade the limit +entirely. This bypass is inherent to plan-cost-based limiting — it +cannot be closed from inside the hook without re-planning under pinned +coefficients — and it is shared with `statement_cost_limit`. The +feature is therefore a cooperative resource-shaping guard, not a +security boundary against an adversary that controls its own session; +the README says so, and the batch-throttling use case must keep it +behind an application-level limit. The `PGC_SUSET` context on the limit +GUCs stops a user *raising the limit* but is not what makes the feature +safe against a hostile role, and must not be mistaken for it. + +The module now keeps per-backend mutable state and registers a +transaction callback it never unregisters (harmless: the library cannot +be unloaded since PG 15). Costs of statements that fail mid-execution +remain charged, which is the intended semantics but can surprise: a +transaction that keeps erroring and retrying under savepoints burns +budget with every attempt. Utility statements that run without a plan +(most DDL) are not counted; those that wrap a query (`CREATE TABLE ... +AS`, `COPY (SELECT ...)`) execute a plan and are counted — the same +statements the per-statement limit has always seen. diff --git a/docs/adr/README.md b/docs/adr/README.md index 906e65b..5dcd999 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -7,3 +7,4 @@ | [0001](0001-support-upstream-supported-majors-only.md) | Support only upstream-supported PostgreSQL majors (currently 14-18) | Accepted | 2026-07-21 | The module targets the PostgreSQL majors with upstream support, enforces a hard floor of 14 at compile time, and drops all pre-13 compatibility scaffolding. | | [0002](0002-behavioral-pg-regress-suite.md) | Behavioral pg_regress suite with version-independent expected output | Accepted | 2026-07-21 | Testing uses a standard pg_regress suite whose output contains no cost numbers or plans, so one expected file serves every supported major on every platform. | | [0003](0003-github-actions-ci-pgdg-matrix.md) | GitHub Actions CI on the pgdg apt matrix, with report-only benchmarks | Accepted | 2026-07-21 | CI builds with -Werror and runs installcheck against pgdg packages for every supported major, runs cppcheck as lint, and publishes pgbench numbers as a non-gating report. | +| [0004](0004-transaction-cost-limit-execution-accounting.md) | Transaction cost limit accumulates at execution time, without savepoint refunds | Accepted | 2026-07-21 | transaction_cost_limit charges each execution's plan cost in an ExecutorStart hook, resets via xact callback, and never refunds rolled-back subtransactions. | diff --git a/docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md b/docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md index 5e64d49..2cb7c2e 100644 --- a/docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md +++ b/docs/superpowers/specs/2026-07-21-transaction-cost-limit-design.md @@ -38,8 +38,11 @@ statement never begins executing): exceeded. Always-accumulate means `SET LOCAL` can disable the check but cannot erase history. -Utility statements never reach the executor and are not counted — the -same statements the per-statement limit has never seen. +Utility statements that run without a plan (most DDL) never reach the +executor and are not counted. Utility statements that wrap a query +(`CREATE TABLE ... AS`, `COPY (SELECT ...)`) do run a plan through the +executor and are counted — the same statements the per-statement limit +has always seen. ## State and reset @@ -71,6 +74,17 @@ a savepoint stack. The suite's post-`LOAD` pin block also pins the new GUC to 0. +## Security boundary + +Both limits compare against the planner's *estimated* cost, which is +computed from `USERSET` cost GUCs (`seq_page_cost`, `cpu_tuple_cost`, +the parallelism costs). A role that can issue `SET` can zero those and +deflate its own estimates below any limit, so this is a cooperative +guard against careless load, not a hard boundary against a hostile +session. The `PGC_SUSET` limit GUCs stop a user *raising the limit* but +do not close this gap. Documented in the README; the batch-throttling +use case keeps it behind an application-level limit. + ## Documentation README section for the new GUC (semantics above, spelled out); ADR 0004 diff --git a/expected/plan_filter.out b/expected/plan_filter.out index b3ee147..2c232c4 100644 --- a/expected/plan_filter.out +++ b/expected/plan_filter.out @@ -10,9 +10,10 @@ SHOW plan_filter.module_loaded; on (1 row) --- Pin both GUCs before doing anything plannable, so the suite also passes --- on a cluster whose configuration preloads the module with a limit set. +-- Pin the GUCs before doing anything plannable, so the suite also passes +-- on a cluster whose configuration preloads the module with limits set. SET plan_filter.statement_cost_limit = 0; +SET plan_filter.transaction_cost_limit = 0; SET plan_filter.filter_select_only = false; CREATE TABLE plan_filter_test AS SELECT g AS x, g % 100 AS y FROM generate_series(1, 10000) g; @@ -71,11 +72,175 @@ UPDATE plan_filter_test SET y = y + 1; ERROR: plan cost limit exceeded HINT: The plan for your query shows that it would probably have an excessive run time. This may be due to a logic error in the SQL, or it may be just a very costly query. Rewrite your query or increase the configuration parameter "plan_filter.statement_cost_limit". ROLLBACK; --- both GUCs are superuser-only: an unprivileged role cannot lift the limit +-- transaction cost limit tests use their own pristine table: earlier tests +-- leave dead tuples in plan_filter_test, and the per-statement cost here +-- must be predictable relative to the limit (one passes, two trip it) +SET plan_filter.statement_cost_limit = 0; +CREATE TABLE plan_filter_xact_test AS + SELECT g AS x, g % 100 AS y FROM generate_series(1, 10000) g; +ANALYZE plan_filter_xact_test; +SET plan_filter.transaction_cost_limit = 300; +-- each statement passes alone, the sum trips the limit +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; + count +------- + 10000 +(1 row) + +SELECT count(*) FROM plan_filter_xact_test; +ERROR: transaction cost limit exceeded +HINT: The combined plan cost of the statements executed so far in this transaction exceeds the configured limit. End the transaction, or increase the configuration parameter "plan_filter.transaction_cost_limit". +COMMIT; +-- the accumulator resets after COMMIT +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; + count +------- + 10000 +(1 row) + +COMMIT; +-- and after an aborted transaction +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; + count +------- + 10000 +(1 row) + +SELECT count(*) FROM plan_filter_xact_test; +ERROR: transaction cost limit exceeded +HINT: The combined plan cost of the statements executed so far in this transaction exceeds the configured limit. End the transaction, or increase the configuration parameter "plan_filter.transaction_cost_limit". +ROLLBACK; +SELECT count(*) FROM plan_filter_xact_test; + count +------- + 10000 +(1 row) + +-- ROLLBACK TO SAVEPOINT does not refund cost already charged +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; + count +------- + 10000 +(1 row) + +SAVEPOINT s; +SELECT count(*) FROM plan_filter_xact_test; +ERROR: transaction cost limit exceeded +HINT: The combined plan cost of the statements executed so far in this transaction exceeds the configured limit. End the transaction, or increase the configuration parameter "plan_filter.transaction_cost_limit". +ROLLBACK TO s; +SELECT count(*) FROM plan_filter_xact_test; +ERROR: transaction cost limit exceeded +HINT: The combined plan cost of the statements executed so far in this transaction exceeds the configured limit. End the transaction, or increase the configuration parameter "plan_filter.transaction_cost_limit". +COMMIT; +-- executions are charged, not plans: a cached generic plan still consumes +SET plan_cache_mode = force_generic_plan; +PREPARE q AS SELECT count(*) FROM plan_filter_xact_test; +BEGIN; +EXECUTE q; + count +------- + 10000 +(1 row) + +EXECUTE q; +ERROR: transaction cost limit exceeded +HINT: The combined plan cost of the statements executed so far in this transaction exceeds the configured limit. End the transaction, or increase the configuration parameter "plan_filter.transaction_cost_limit". +COMMIT; +DEALLOCATE q; +RESET plan_cache_mode; +-- Plain EXPLAIN charges nothing. The two real SELECTs total ~378; with a +-- 500 limit they both pass only if the three EXPLAINs between them add zero. +-- Were EXPLAIN charged -- even if it stayed exempt from the check itself -- +-- the trailing SELECT would cross 500 and trip. +SET plan_filter.transaction_cost_limit = 500; +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; + count +------- + 10000 +(1 row) + +EXPLAIN (COSTS OFF) SELECT count(*) FROM plan_filter_xact_test; + QUERY PLAN +----------------------------------------- + Aggregate + -> Seq Scan on plan_filter_xact_test +(2 rows) + +EXPLAIN (COSTS OFF) SELECT count(*) FROM plan_filter_xact_test; + QUERY PLAN +----------------------------------------- + Aggregate + -> Seq Scan on plan_filter_xact_test +(2 rows) + +EXPLAIN (COSTS OFF) SELECT count(*) FROM plan_filter_xact_test; + QUERY PLAN +----------------------------------------- + Aggregate + -> Seq Scan on plan_filter_xact_test +(2 rows) + +SELECT count(*) FROM plan_filter_xact_test; + count +------- + 10000 +(1 row) + +COMMIT; +-- filter_select_only exempts non-SELECT statements from the transaction +-- accounting too. One UPDATE plans at ~164; with the limit at 100 an exempt +-- UPDATE passes (charging nothing, so two in a row also pass), while the same +-- UPDATE charged trips the limit on its own. +SET plan_filter.transaction_cost_limit = 100; +SET plan_filter.filter_select_only = true; +BEGIN; +UPDATE plan_filter_xact_test SET y = y; +UPDATE plan_filter_xact_test SET y = y; +ROLLBACK; +SET plan_filter.filter_select_only = false; +BEGIN; +UPDATE plan_filter_xact_test SET y = y; +ERROR: transaction cost limit exceeded +HINT: The combined plan cost of the statements executed so far in this transaction exceeds the configured limit. End the transaction, or increase the configuration parameter "plan_filter.transaction_cost_limit". +ROLLBACK; +SET plan_filter.transaction_cost_limit = 0; +DROP TABLE plan_filter_xact_test; +-- Both limits raise SQLSTATE 54001 (statement_too_complex) -- the contract +-- PostgREST maps to HTTP 429. Assert it symbolically, via the condition +-- name, so a reworded message can never mask an errcode change; the handler +-- only catches the block if the SQLSTATE still matches. +SET plan_filter.statement_cost_limit = 1; +DO $$ +BEGIN + PERFORM count(*) FROM pg_class; + RAISE EXCEPTION 'statement_cost_limit did not fire'; +EXCEPTION WHEN statement_too_complex THEN + RAISE NOTICE 'statement_cost_limit raises SQLSTATE %', SQLSTATE; +END $$; +NOTICE: statement_cost_limit raises SQLSTATE 54001 +SET plan_filter.statement_cost_limit = 0; +SET plan_filter.transaction_cost_limit = 1; +DO $$ +BEGIN + PERFORM count(*) FROM pg_class; + RAISE EXCEPTION 'transaction_cost_limit did not fire'; +EXCEPTION WHEN statement_too_complex THEN + RAISE NOTICE 'transaction_cost_limit raises SQLSTATE %', SQLSTATE; +END $$; +NOTICE: transaction_cost_limit raises SQLSTATE 54001 +SET plan_filter.transaction_cost_limit = 0; +-- all three GUCs are superuser-only: an unprivileged role cannot lift them CREATE ROLE regress_plan_filter_user; SET ROLE regress_plan_filter_user; SET plan_filter.statement_cost_limit = 0; ERROR: permission denied to set parameter "plan_filter.statement_cost_limit" +SET plan_filter.transaction_cost_limit = 0; +ERROR: permission denied to set parameter "plan_filter.transaction_cost_limit" SET plan_filter.filter_select_only = true; ERROR: permission denied to set parameter "plan_filter.filter_select_only" RESET ROLE; diff --git a/plan_filter.c b/plan_filter.c index cde347b..04c9da6 100644 --- a/plan_filter.c +++ b/plan_filter.c @@ -18,6 +18,9 @@ #include +#include "access/parallel.h" +#include "access/xact.h" +#include "executor/executor.h" #include "optimizer/planner.h" #include "utils/guc.h" @@ -37,15 +40,29 @@ PG_MODULE_MAGIC; static double statement_cost_limit = 0.0; +static double transaction_cost_limit = 0.0; + static bool module_loaded = false; static bool filter_select_only = false; +/* + * Estimated cost accumulated by the statements executed so far in the + * current transaction. Always maintained (even with no limit set), so + * that enabling the limit mid-transaction still sees the full history; + * zeroed by plan_filter_xact_callback at end of transaction. + */ +static double xact_cost_total = 0.0; + static planner_hook_type prev_planner_hook = NULL; +static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL; + static PlannedStmt *limit_func(Query *parse, const char *query_string, int cursorOptions, ParamListInfo boundParams); +static void xact_limit_func(QueryDesc *queryDesc, int eflags); +static void plan_filter_xact_callback(XactEvent event, void *arg); void _PG_init(void); @@ -73,6 +90,20 @@ _PG_init(void) NULL, NULL); + /* Define custom GUC variable. */ + DefineCustomRealVariable("plan_filter.transaction_cost_limit", + "Sets the maximum allowed total plan cost of " + "statements executed in one transaction.", + "Zero turns this feature off.", + &transaction_cost_limit, + 0.0, + 0.0, DBL_MAX, + PGC_SUSET, + 0, /* no flags required */ + NULL, + NULL, + NULL); + /* Define custom GUC variable. */ DefineCustomBoolVariable("plan_filter.module_loaded", "true if the module is loaded ", @@ -110,9 +141,13 @@ _PG_init(void) EmitWarningsOnPlaceholders("plan_filter"); #endif - /* install the hook */ + /* install the hooks */ prev_planner_hook = planner_hook; planner_hook = limit_func; + prev_ExecutorStart_hook = ExecutorStart_hook; + ExecutorStart_hook = xact_limit_func; + + RegisterXactCallback(plan_filter_xact_callback, NULL); } static PlannedStmt * @@ -146,3 +181,71 @@ limit_func(Query *parse, const char *query_string, int cursorOptions, return result; } + +/* + * ExecutorStart hook: charge each execution's plan cost against the + * transaction's budget. + * + * Accounting happens here rather than in the planner hook so that every + * execution is counted: repeated EXECUTE of a prepared statement stops + * invoking the planner once a generic plan is cached, which would let a + * PREPARE/EXECUTE loop evade a plan-time accumulator entirely. + * + * The check runs before chaining to the rest of ExecutorStart, so a + * statement that busts the budget never begins executing. + */ +static void +xact_limit_func(QueryDesc *queryDesc, int eflags) +{ + /* + * Plain EXPLAIN neither consumes budget nor is refused by an exhausted + * one; it stays usable for inspecting plans. Parallel workers are + * excluded because the leader already counted the plan. + */ + if (!(eflags & EXEC_FLAG_EXPLAIN_ONLY) && + !IsParallelWorker() && + !(filter_select_only && queryDesc->operation != CMD_SELECT)) + { + xact_cost_total += queryDesc->plannedstmt->planTree->total_cost; + + if (transaction_cost_limit > 0.0 && + xact_cost_total > transaction_cost_limit) + ereport(ERROR, + (errcode(ERRCODE_STATEMENT_TOO_COMPLEX), + errmsg("transaction cost limit exceeded"), + errhint("The combined plan cost of the statements " + "executed so far in this transaction exceeds the " + "configured limit. End the transaction, or " + "increase the configuration parameter " + "\"plan_filter.transaction_cost_limit\"."))); + } + + if (prev_ExecutorStart_hook) + (*prev_ExecutorStart_hook) (queryDesc, eflags); + else + standard_ExecutorStart(queryDesc, eflags); +} + +/* + * Zero the transaction cost accumulator at end of transaction. These + * callbacks may not raise errors; a plain assignment is safe. There is + * deliberately no subtransaction callback: ROLLBACK TO SAVEPOINT does not + * refund cost already charged, since the limit guards resource + * consumption and the work was attempted. + */ +static void +plan_filter_xact_callback(XactEvent event, void *arg) +{ + switch (event) + { + case XACT_EVENT_COMMIT: + case XACT_EVENT_PARALLEL_COMMIT: + case XACT_EVENT_ABORT: + case XACT_EVENT_PARALLEL_ABORT: + case XACT_EVENT_PREPARE: + xact_cost_total = 0.0; + break; + default: + break; + } +} diff --git a/sql/plan_filter.sql b/sql/plan_filter.sql index 5e6c442..b59dfbd 100644 --- a/sql/plan_filter.sql +++ b/sql/plan_filter.sql @@ -6,9 +6,10 @@ LOAD 'plan_filter'; SHOW plan_filter.module_loaded; --- Pin both GUCs before doing anything plannable, so the suite also passes --- on a cluster whose configuration preloads the module with a limit set. +-- Pin the GUCs before doing anything plannable, so the suite also passes +-- on a cluster whose configuration preloads the module with limits set. SET plan_filter.statement_cost_limit = 0; +SET plan_filter.transaction_cost_limit = 0; SET plan_filter.filter_select_only = false; CREATE TABLE plan_filter_test AS @@ -53,10 +54,112 @@ BEGIN; UPDATE plan_filter_test SET y = y + 1; ROLLBACK; --- both GUCs are superuser-only: an unprivileged role cannot lift the limit +-- transaction cost limit tests use their own pristine table: earlier tests +-- leave dead tuples in plan_filter_test, and the per-statement cost here +-- must be predictable relative to the limit (one passes, two trip it) +SET plan_filter.statement_cost_limit = 0; +CREATE TABLE plan_filter_xact_test AS + SELECT g AS x, g % 100 AS y FROM generate_series(1, 10000) g; +ANALYZE plan_filter_xact_test; +SET plan_filter.transaction_cost_limit = 300; + +-- each statement passes alone, the sum trips the limit +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; +SELECT count(*) FROM plan_filter_xact_test; +COMMIT; + +-- the accumulator resets after COMMIT +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; +COMMIT; + +-- and after an aborted transaction +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; +SELECT count(*) FROM plan_filter_xact_test; +ROLLBACK; +SELECT count(*) FROM plan_filter_xact_test; + +-- ROLLBACK TO SAVEPOINT does not refund cost already charged +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; +SAVEPOINT s; +SELECT count(*) FROM plan_filter_xact_test; +ROLLBACK TO s; +SELECT count(*) FROM plan_filter_xact_test; +COMMIT; + +-- executions are charged, not plans: a cached generic plan still consumes +SET plan_cache_mode = force_generic_plan; +PREPARE q AS SELECT count(*) FROM plan_filter_xact_test; +BEGIN; +EXECUTE q; +EXECUTE q; +COMMIT; +DEALLOCATE q; +RESET plan_cache_mode; + +-- Plain EXPLAIN charges nothing. The two real SELECTs total ~378; with a +-- 500 limit they both pass only if the three EXPLAINs between them add zero. +-- Were EXPLAIN charged -- even if it stayed exempt from the check itself -- +-- the trailing SELECT would cross 500 and trip. +SET plan_filter.transaction_cost_limit = 500; +BEGIN; +SELECT count(*) FROM plan_filter_xact_test; +EXPLAIN (COSTS OFF) SELECT count(*) FROM plan_filter_xact_test; +EXPLAIN (COSTS OFF) SELECT count(*) FROM plan_filter_xact_test; +EXPLAIN (COSTS OFF) SELECT count(*) FROM plan_filter_xact_test; +SELECT count(*) FROM plan_filter_xact_test; +COMMIT; + +-- filter_select_only exempts non-SELECT statements from the transaction +-- accounting too. One UPDATE plans at ~164; with the limit at 100 an exempt +-- UPDATE passes (charging nothing, so two in a row also pass), while the same +-- UPDATE charged trips the limit on its own. +SET plan_filter.transaction_cost_limit = 100; +SET plan_filter.filter_select_only = true; +BEGIN; +UPDATE plan_filter_xact_test SET y = y; +UPDATE plan_filter_xact_test SET y = y; +ROLLBACK; +SET plan_filter.filter_select_only = false; +BEGIN; +UPDATE plan_filter_xact_test SET y = y; +ROLLBACK; + +SET plan_filter.transaction_cost_limit = 0; +DROP TABLE plan_filter_xact_test; + +-- Both limits raise SQLSTATE 54001 (statement_too_complex) -- the contract +-- PostgREST maps to HTTP 429. Assert it symbolically, via the condition +-- name, so a reworded message can never mask an errcode change; the handler +-- only catches the block if the SQLSTATE still matches. +SET plan_filter.statement_cost_limit = 1; +DO $$ +BEGIN + PERFORM count(*) FROM pg_class; + RAISE EXCEPTION 'statement_cost_limit did not fire'; +EXCEPTION WHEN statement_too_complex THEN + RAISE NOTICE 'statement_cost_limit raises SQLSTATE %', SQLSTATE; +END $$; +SET plan_filter.statement_cost_limit = 0; + +SET plan_filter.transaction_cost_limit = 1; +DO $$ +BEGIN + PERFORM count(*) FROM pg_class; + RAISE EXCEPTION 'transaction_cost_limit did not fire'; +EXCEPTION WHEN statement_too_complex THEN + RAISE NOTICE 'transaction_cost_limit raises SQLSTATE %', SQLSTATE; +END $$; +SET plan_filter.transaction_cost_limit = 0; + +-- all three GUCs are superuser-only: an unprivileged role cannot lift them CREATE ROLE regress_plan_filter_user; SET ROLE regress_plan_filter_user; SET plan_filter.statement_cost_limit = 0; +SET plan_filter.transaction_cost_limit = 0; SET plan_filter.filter_select_only = true; RESET ROLE; DROP ROLE regress_plan_filter_user;