fix(security): close eight PII-denylist bypasses (v0.2.0)

.github/workflows/ci.yml

@@ -11,7 +11,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.10", "3.11", "3.12", "3.13"]
+        python-version: ["3.11", "3.12", "3.13"]
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5

CHANGELOG.md

@@ -7,6 +7,176 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.2.0] — 2026-08-17
+
+Security release. Closes eight PII-denylist bypasses: two found by adversarial
+review of a deployed agent, and six more found while fixing those. Every one
+was confirmed against 0.1.1 with a reproducing query before being fixed, and
+each has a regression test. **Contains breaking behaviour changes** — queries
+that 0.1.1 allowed are now denied. That is the point of the release; see
+*Upgrading*.
+
+The most severe was not in the original report: `SELECT c FROM tbl AS c`
+returned every column of every row, PII included, and the guard auto-executed
+it.
+
+### Security
+- **PII denylist is now enforced in every query scope, not just the outermost
+  select list.** `PiiProjectionRule` called `outermost_projection_names`, so
+  any inner scope that renamed a denied column laundered it past the guard.
+  All of these returned `confirm` on 0.1.1 and are denied as of 0.2.0:
+
+  ```sql
+  WITH c AS (SELECT BillingCity AS city FROM `p.d.t`) SELECT city FROM c
+  SELECT x FROM (SELECT BillingCity AS x FROM `p.d.t`)
+  SELECT uid FROM `p.d.t` UNION ALL SELECT c FROM (SELECT email AS c FROM `p.d.t`)
+  WITH a AS (SELECT email AS e1 FROM `p.d.t`), b AS (SELECT e1 AS e2 FROM a) SELECT e2 FROM b
+  ```
+
+  Matching applies to the underlying column names within each scope, so an
+  alias never launders a denied column.
+- **Denied columns can no longer be used to probe values.** The denylist gated
+  projection only, so `WHERE BillingCity = 'Columbus'`, `GROUP BY`, `HAVING`
+  and `ORDER BY` references all passed — none return the column, but each
+  answers a yes/no question about its value, and enough queries reconstruct it.
+  The new `pii_mode` config switch denies *any* reference by default.
+- **`SELECT *` is rejected in every scope,** not just the outermost one. Once
+  PII checking is all-scope, a star inside a CTE or derived table makes that
+  scope's projection list unresolvable, so the guard cannot prove a denied
+  column is absent — same rationale as the existing top-level rule.
+- **Qualified `t.*` is now caught.** It parses as an `exp.Column` wrapping a
+  `Star`, which the previous `isinstance(projection, exp.Star)` check missed,
+  so `SELECT t.* FROM tbl t` bypassed the star rule even at the top level.
+- **Whole-row references by row-source alias are rejected.** `SELECT c FROM tbl AS c`
+  returns every column in the row as a struct — strictly more powerful than the
+  `SELECT *` the guard already blocked, and it parsed as an ordinary column
+  named `c`, so the denylist had nothing to match. Also covers
+  `TO_JSON_STRING(c)`, `ARRAY_AGG(c)`, `STRUCT(c)`, the unaliased
+  `SELECT tbl FROM tbl` form, CTE names, derived-table aliases
+  (`SELECT d FROM (SELECT ...) AS d`), and `VALUES` / `PIVOT` aliases.
+  New `NoUnresolvableColumnsRule`.
+
+  The rule resolves ambiguity from the AST rather than denying on a bare name
+  collision, which would have made it unusable: a table contributes only the
+  name it is addressable by (an aliased `p.d.status AS s` does not reserve
+  `status`), and a CTE or derived table publishes its own output names, so
+  `WITH revenue AS (SELECT ..., SUM(x) AS revenue ...) SELECT revenue FROM revenue`
+  — a mainstream idiom — is correctly read as a column reference.
+- **ClickHouse `COLUMNS('regex')` is rejected.** It expands to many columns but
+  parses with no `Star` node at all, so a star check keyed on `exp.Star` alone
+  never saw it.
+- **`NATURAL JOIN` is rejected.** It joins on whichever columns the tables
+  share — a schema fact the guard does not have, so it cannot rule out a denied
+  column among them. `JOIN ... USING (...)` remains allowed and is now read.
+- **Stars nested inside a wrapping construct are caught.** The star check only
+  inspected the projection's root node, so `OBJECT_CONSTRUCT(*)` (Snowflake),
+  `COLUMNS(*)` (DuckDB), `* APPLY(f)` (ClickHouse) and `ROW(c.*)` (Trino) all
+  passed. It is now a deep walk, with `COUNT(*)` / `COUNT(DISTINCT *)` as the
+  explicit carve-out.
+- **Column names that never become an `exp.Column` are now read.** sqlglot
+  parses several positions as bare `exp.Identifier`, so reference mode's
+  `find_all(exp.Column)` sweep missed them: `JOIN ... USING (email)`,
+  `... AS g(email)` column aliases, and `STRUCT('x' AS email)` field names.
+  `JOIN ... USING (email)` was a working single-query value oracle.
+- **Aggregation is no longer a blanket PII exemption.** Every `exp.AggFunc`
+  counted as PII-neutralising, so `MAX(email)`, `MIN(email)`,
+  `ARRAY_AGG(email)`, `STRING_AGG(email)` and `ANY_VALUE(email)` returned real
+  values through `pii_mode="project"`. Only aggregates that reduce to a derived
+  statistic (`COUNT`, `COUNTIF`, `SUM`, `AVG`, `APPROX_COUNT_DISTINCT`,
+  `STDDEV`, `VARIANCE`) qualify now.
+
+### Added
+- `pii_mode` on `SqlGuardConfig` and `SqlGuardConfig.from_settings`, typed as
+  `PiiMode = Literal["reference", "project"]`. Defaults to `"reference"`
+  (deny any reference to a denied column). `"project"` restores 0.1.1-style
+  projection-only checking, still applied across all scopes. An invalid value
+  raises `ValueError` rather than silently falling back to a default.
+- `NoSelectStarRule` — the all-scope star rule. `NoTopLevelStarRule` remains
+  importable as an alias of it.
+- `NoUnresolvableColumnsRule` — rejects whole-row table-alias references and
+  `NATURAL JOIN`. Added to `default_rules` between the star rule and the PII
+  rule.
+- AST helpers for custom rules: `all_selects`, `all_projection_names`,
+  `all_referenced_column_names`, `has_select_star`, `whole_row_references`.
+
+### Changed
+- `default_rules` substitutes `NoSelectStarRule` for `NoTopLevelStarRule`
+  (the same object under its new name). Ordering semantics are unchanged: the
+  star rule still runs before `PiiProjectionRule`, which now also guarantees a
+  star is rejected before the PII check tries to resolve a projection list it
+  cannot see.
+- `outermost_projection_names` and `has_top_level_select_star` are retained
+  and still outermost-only; their docstrings now warn against using them for
+  denylist enforcement. `has_top_level_select_star` does now recognise `t.*`.
+- **Minimum Python is now 3.11.** The package has imported `enum.StrEnum`
+  (3.11+) since 0.1.0 while advertising `requires-python = ">=3.10"`, so
+  `import sql_guard` raised `ImportError` on 3.10 and that CI matrix job could
+  never have passed. `requires-python`, the classifiers, ruff's
+  `target-version`, mypy's `python_version` and the CI matrix now agree on
+  3.11. No source change was needed — this documents what the code already
+  required.
+
+### Fixed
+- `format_cost` no longer wraps `math.floor` in a redundant `int()`
+  (`math.floor` already returns an `int`). Behaviour is unchanged; this clears
+  a `ruff check` failure.
+- `_projection_names` reads `.this` instead of calling sqlglot's untyped
+  `unalias()`. Identical result — `unalias` returns `self.this` for an `Alias`
+  — but it clears the only `mypy --strict` error in the package.
+
+### Known limits (documented, not fixed)
+
+The denylist matches column names in the SQL text and the guard has no schema,
+so these remain out of scope and are now stated explicitly in the README:
+PII inside JSON/VARIANT/STRUCT payloads addressed by string literals
+(`JSON_VALUE(payload, '$.email')`), re-identification through non-PII columns,
+and side channels such as row counts and dry-run byte counts.
+
+Also uncovered: selecting a STRUCT column whole, and `UNNEST` aliases over an
+array of structs — both return every field without naming one, and neither is
+distinguishable at parse time from the legitimate scalar-array form.
+
+Separately: several entries in `_PII_SAFE_FUNC_KEYS` (`sha256`,
+`farm_fingerprint`) match no sqlglot key and never fire, so those spellings of
+hashed PII are *denied* rather than treated as neutralised. `md5` does match
+via the `TO_HEX(MD5(x))` folding, so that spelling is permitted under
+`pii_mode="project"` — the two are inconsistent. Left as-is because the
+inconsistency errs toward denial; revisit deliberately when deciding whether
+hashed PII is acceptable output.
+
+### Known issues (unfixed, pre-existing)
+
+- `SelectOnlyRule` accepts `exp.Select | exp.Union`, but sqlglot 26.x derives
+  `Except` and `Intersect` from `SetOperation` rather than `Union`, so a
+  top-level `EXCEPT DISTINCT` / `INTERSECT` is rejected as "Only SELECT
+  statements are allowed". Fails closed, so it is an availability bug rather
+  than a security one, but it makes the `Except`/`Intersect` branch of
+  `outer_selects` dead code for top-level set operations.
+- `AllowedTablesRule` runs last, so a query that is both off-allowlist and
+  star/PII-violating is always reported as the latter. Telemetry built on
+  `decision.reason` will under-count allowlist breaches.
+
+### Upgrading
+
+Queries denied by 0.2.0 that 0.1.1 allowed fall into two groups:
+
+1. **Bypasses** — alias laundering, inner-scope and nested stars, whole-row
+   aliases, `NATURAL JOIN`, identifier-only column references, and PII through
+   value-preserving aggregates. There is no supported way to re-permit these,
+   by design. Whole-row alias references are fixed by qualifying them
+   (`SELECT c.col` rather than `SELECT c`).
+2. **Denied columns in predicates** — set `pii_mode="project"` to restore the
+   old behaviour. Understand that this re-opens the value-probing oracle
+   described above; prefer narrowing the denylist or exposing a pre-masked
+   view.
+
+The bundled `Q1` identity-resolution query (normalising `email`/`mobile` in a
+CTE, projecting only `COUNTIF` aggregates) is denied in **both** modes: the CTE
+scope projects the denied columns, and `COUNTIF(email_norm = 'target')` is
+itself a value oracle. Aggregates over denied columns are not safe under this
+threat model. Move such normalisation into a warehouse view the denylist does
+not cover.
+
 ## [0.1.1] — 2026-06-19
 
 ### Added
@@ -51,6 +221,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `SELECT * EXCEPT(...)` is rejected — the guard cannot prove the EXCEPT
   list enumerates every PII column.
 
-[Unreleased]: https://github.com/sakura-sky/sql-guard/compare/v0.1.1...HEAD
+[Unreleased]: https://github.com/sakura-sky/sql-guard/compare/v0.2.0...HEAD
+[0.2.0]: https://github.com/sakura-sky/sql-guard/compare/v0.1.1...v0.2.0
 [0.1.1]: https://github.com/sakura-sky/sql-guard/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/sakura-sky/sql-guard/releases/tag/v0.1.0

README.md

@@ -33,19 +33,104 @@ if decision.denied:
 
 1. **Single SELECT only.** DML, DDL, scripts, multi-statement payloads — all
    rejected. Even if buried in subqueries.
-2. **PII column denylist.** Projections that touch denylisted columns are
-   rejected. Catches aliased PII (`SELECT email AS x`), PII through transforms
-   (`SELECT LOWER(email)`), and the right arm of a `UNION ALL`.
-3. **No top-level `SELECT *`.** Bare `*`, `* EXCEPT(...)`, and `* REPLACE(...)`
-   are all rejected — the guard can't prove EXCEPT enumerates every PII column.
-4. **Table allowlist.** Only fully-qualified tables you approved can be
+2. **PII column denylist.** By default, *any reference* to a denylisted column
+   is rejected — in the select list, `WHERE`, `GROUP BY`, `HAVING`, `ORDER BY`
+   or a `JOIN` condition, at any nesting depth. Catches aliased PII
+   (`SELECT email AS x`), PII through transforms (`SELECT LOWER(email)`),
+   the right arm of a `UNION ALL`, and columns renamed inside a CTE or derived
+   table. See [PII modes](#pii-modes).
+3. **No `SELECT *` in any scope.** Bare `*`, `* EXCEPT(...)`, `* REPLACE(...)`
+   and qualified `t.*` are all rejected, inside CTEs and subqueries as well as
+   at the top level — the guard can't prove EXCEPT enumerates every PII column,
+   nor what `*` expands to. The check is a deep walk, so stars wrapped in a
+   function (`OBJECT_CONSTRUCT(*)`, `COLUMNS(*)`, `* APPLY(f)`) are caught too;
+   `COUNT(*)` is the deliberate exception.
+4. **Nothing whose columns the guard can't enumerate.** A bare table alias in a
+   value position (`SELECT c FROM tbl AS c`) returns the whole row as a struct,
+   and `NATURAL JOIN` joins on unknown shared columns. Both are rejected for the
+   same reason as `SELECT *`.
+5. **Table allowlist.** Only fully-qualified tables you approved can be
    referenced. CTE aliases are excluded.
-5. **Cost cap.** Given the bytes-processed figure from a dry-run, the guard
+6. **Cost cap.** Given the bytes-processed figure from a dry-run, the guard
    returns `allow` below your auto threshold, `confirm` in between, and `deny`
    above the hard cap or bytes-billed ceiling.
 
 Every check is a `Rule` you can replace or compose with.
 
+### PII modes
+
+`pii_mode` controls how far the denylist reaches.
+
+| Mode | Denies | Use when |
+|---|---|---|
+| `"reference"` (default) | Any reference to a denied column, in any clause and any scope. | The agent must not learn PII values at all. |
+| `"project"` | Only projections of denied columns — checked in every scope. | Predicate access to PII is a deliberate, accepted trade-off. |
+
+The default is the strict one because projection-only checking leaves the
+values reachable. A denied column in a `WHERE` clause never appears in the
+output, but the row count still answers a yes/no question about it:
+
+```sql
+-- Passes a projection-only guard. Returns 0 or non-zero.
+SELECT COUNT(*) FROM `p.d.orders` WHERE billing_city = 'Columbus'
+```
+
+Repeat with `LIKE 'a%'`, `> 'm'`, and so on, and the value falls out in a
+handful of queries. `GROUP BY`, `HAVING` and `ORDER BY` leak the same way.
+
+**The loosening path.** If your deployment genuinely needs to filter on PII —
+segmenting on a hashed identifier, say, or counting non-null contact rows —
+set `pii_mode="project"`:
+
+```python
+SqlGuardConfig.from_settings(..., pii_mode="project")
+```
+
+That re-permits denied columns in predicates while still rejecting every
+projection of them, in every scope. Prefer narrowing the denylist, or exposing
+a pre-masked warehouse view the denylist doesn't cover, before reaching for it.
+
+Note that "aggregate" is not a safe harbour in either mode. `MAX(email)`,
+`ARRAY_AGG(email)` and `STRING_AGG(email)` return real values and are rejected;
+only aggregates that reduce to a statistic (`COUNT`, `SUM`, `AVG`, `STDDEV`, …)
+are treated as PII-neutralising, and then only under `pii_mode="project"`.
+
+### What the denylist does not cover
+
+The guard matches **column names in the SQL text**. It has no schema, so:
+
+- **PII inside JSON / VARIANT / STRUCT payloads** is not covered.
+  `JSON_VALUE(payload, '$.email')` names only `payload`; the field name is a
+  string literal the engine resolves. The same applies to selecting a struct
+  column whole (`SELECT contact FROM t`) and to `UNNEST` aliases over an array
+  of structs (`SELECT s FROM t, UNNEST(t.contacts) AS s`) — both return every
+  field without naming one. **Denylist the containing column.**
+- **Re-identification through non-PII columns** is out of scope. If `uid` maps
+  1:1 to a person, blocking `email` does not prevent correlation with outside
+  data.
+- **Side channels** — row counts, dry-run byte counts and error messages carry
+  bits about denied values even when every direct reference is refused.
+
+These are limits of a parse-level guard, not bugs. Warehouse-side column
+security is the durable answer; `sql-guard` is defence in depth.
+
+### Scopes
+
+Denylist and star checks run against **every** `SELECT` scope — CTE bodies,
+derived tables, scalar and `IN` subqueries, and each arm of a set operation —
+matching on the underlying column names in the scope that names them. An alias
+therefore cannot launder a denied column:
+
+```sql
+-- Denied: the CTE scope still names billing_city.
+WITH c AS (SELECT billing_city AS city FROM `p.d.orders`)
+SELECT city FROM c
+```
+
+Checking only the outermost select list would see `city` and let it through.
+The same applies through derived tables, `UNION` arms, and multi-hop alias
+chains (`a AS (...) → b AS (...) → SELECT`).
+
 ### The cost-cap rule in detail
 
 Three independent thresholds bound any single query:
@@ -70,7 +155,8 @@ through — useful for sandboxes or onboarding a new tenant.
   job; pass `bytes_processed` to `evaluate_cost`. Keeps the guard testable
   without credentials and dialect-agnostic.
 - It does not introspect table schemas. If you say "this table is allowed,"
-  the guard takes your word for it.
+  the guard takes your word for it. This is why `SELECT *` is rejected
+  everywhere: without a schema the guard cannot enumerate what `*` returns.
 - It does not authorise the user. Identity, IAM, row-level security: not in
   scope. The guard is a *policy* layer, not a *permissions* layer.
 
@@ -159,7 +245,7 @@ pip install 'sql-guard[adk]'     # + Google ADK + BigQuery client for the
                                  #   FunctionTool integration
 ```
 
-Python 3.10+ supported.
+Python 3.11+ supported.
 
 ## License
 

pyproject.toml

@@ -1,9 +1,9 @@
 [project]
 name = "sql-guard"
-version = "0.1.1"
+version = "0.2.0"
 description = "Deterministic policy engine for LLM-generated SQL: SELECT-only, PII column denylist, table allowlist, cost cap. Multi-dialect via sqlglot (BigQuery, Snowflake, Postgres, Trino, …)."
 readme = "README.md"
-requires-python = ">=3.10"
+requires-python = ">=3.11"
 license = { text = "Apache-2.0" }
 authors = [{ name = "Sakura Sky Engineering" }]
 keywords = [
@@ -24,7 +24,6 @@ classifiers = [
     "License :: OSI Approved :: Apache Software License",
     "Operating System :: OS Independent",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
     "Programming Language :: Python :: 3.13",
@@ -64,11 +63,11 @@ packages = ["src/sql_guard"]
 
 [tool.ruff]
 line-length = 100
-target-version = "py310"
+target-version = "py311"
 src = ["src", "tests"]
 
 [tool.mypy]
-python_version = "3.10"
+python_version = "3.11"
 strict = true
 warn_return_any = true
 warn_unused_configs = true