fix(gfql): apply GRAPH residual predicates as masks

CHANGELOG.md

@@ -30,6 +30,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 - **GFQL Cypher parser: Earley → LALR(1) (~70× faster parse, same AST)**: `parse_cypher` now uses a single LALR(1) parser (contextual lexer) instead of Earley (dynamic lexer) — ~0.25ms vs ~17ms per query on representative queries. The WHERE grammar was unified to one `where_clause: "WHERE" expr` rule (the former `where_predicates | expr` dual rule was a genuine reduce/reduce ambiguity that forced Earley), and the structured flat-AND predicate form is recovered by a post-parse lift that re-parses the WHERE body with a LALR sub-parser (`start="where_predicate_chain"`). The full WHERE language is preserved — OR/XOR/NOT, parentheses, arithmetic, IN, pattern predicates — nothing is restricted to a subset. The grammar was then **purified to (near-)unambiguous**: WHERE binds to its preceding clause *in the grammar* (bundled into `match_clause`; the WITH..WHERE attachment ambiguity is eliminated, not tie-broken), and name-rooted dot chains derive only via `qualified_name` (the `property_access` redundancy is gone) — dropping the redundant `label_predicate_expr` rule (`(n:Admin)` parses via `grouped_expr`), and excluding a top-level `IN` from list-literal elements (`[x IN xs ...` is comprehension syntax; allowing a bare `IN` list element made it overlap). Net: the LALR conflict profile goes from 8 shift/reduce to **ZERO conflicts** — the grammar is now **provably unambiguous LALR(1)** and builds under Lark's `strict=True` (a build-time proof, machine-checked in CI as zero conflicts, plus a strict-mode build test where the optional `interegular` dep is present). Every input has a single derivation. **Machine-checked invariants** (`test_grammar_invariants.py`): (1) **zero LALR conflicts** (dependency-free, always-on in CI) plus a `strict=True` build test (skipped where the optional `interegular` dep is absent) — a grammar edit that introduces any ambiguity fails CI; (2) semantic ambiguity is ZERO — every corpus query has exactly one Earley derivation and it transforms to a single AST, no exceptions; (3) a rule-coverage gate (`test_every_grammar_rule_is_exercised_by_the_corpus`) forces every new grammar rule through the invariants; (4) differential tests run the production pipeline under both parsers (byte-identical ASTs, identical rejections). **Full-repo differentials** (1,850+ scraped queries, LALR-vs-Earley and old-vs-new production): **zero AST divergences**, and a small set of deliberate language fixes — accept-by-accident shapes with ill-defined semantics, now honest syntax errors and pinned as tests (`DELIBERATE_LANGUAGE_FIXES`): `RETURN DISTINCT` with no items (Earley re-lexed DISTINCT as a variable name), double WHERE (the old positional attachment kept *both* predicates in different AST fields), double post-WITH WHERE, WHERE after UNWIND, WHERE before MATCH in a graph constructor, and the invalid "list of IN-booleans" (`[x IN xs, y]`; use `[(x IN xs), y]` for a genuine bool list). The lift stays an internal optimization: only a flat `AND` chain over present columns lifts to `filter_dict`; parens / OR / XOR / NOT and any absent-column case stay on `where_rows` (a correctness boundary, since `where_rows` treats an absent property as null while `filter_dict` would raise). Full cypher suite (1,681 tests) passes. Downstream execution (`filter_dict` vs `where_rows` routing) is unchanged. A pre-existing `<>`-over-null 3VL divergence between the two execution paths (surfaced by #1653's metamorphic test) is tracked separately in #1683.
 
 ### Fixed
+- **GFQL Cypher `GRAPH { }` residual predicates now fail safely or apply as graph masks**: `GRAPH { MATCH ... WHERE ... }` no longer silently drops predicates that the graph-state path cannot apply. Safe one-node/one-edge residual filters, including disjunctions and `searchAny(...)`, are applied before graph-state matching; unsupported pattern-predicate, multi-alias, and Polars graph-residual cases now raise clear validation errors instead of returning an over-broad subgraph.
 - **GFQL Cypher `=~` / scalar-fn cross-engine hardening (review wave, dgx-verified)**: (1) composed `=~` (`WHERE … =~ … OR …`, `RETURN`-expression position) now works on `engine='cudf'` — the series evaluator used raw `.str.fullmatch`, which cuDF lacks, and the resulting `AttributeError` was masked as "unsupported predicate op" (it now routes through the `Fullmatch` predicate's engine workarounds, and honest `NotImplementedError` declines pass through instead of being re-labeled); (2) the cuDF fullmatch emulation anchors alternations as a whole (`^(ab|cd)$` — bare `^ab|cd$` silently matched `'abXXX'`); (3) the cuDF case-insensitive `(?i)` lowercase-folding workaround now declines the fold-unsound shapes — uppercase escape classes (`.lower()` turns `\D` into `\d`, silently inverting the predicate), case-crossing character ranges (`(?i)[A-z]` silently narrowed; `[X-b]` folded to an invalid range), and non-ASCII patterns — while lowercase escapes (`\d`, `\.`) keep folding as before; lookaround, backreferences, and named-group refs decline up front (libcudf rejects them at kernel-compile time); (4) polars `Match`/`Fullmatch` lowering applies the same Rust-regex guard as `Contains` (lookaround/backrefs decline instead of a non-NIE `ComputeError` at collect); (5) `toLower`/`toUpper` on a non-string column decline like neo4j's type error instead of broadcasting the stringified Series repr (pandas/cuDF) or raising a non-NIE `SchemaError` (polars-gpu); (6) polars `floor`/`ceil`/`round` cast to `Float64` so integer columns return Float like neo4j and the pandas engine; (7) invalid regex patterns on the composed path raise a clear "invalid regex pattern" error instead of "unsupported predicate op".
 - **GFQL Cypher `round()` hardening (review wave, dgx-verified)**: (1) the `polars` extra's floor is now `polars>=1.29` — `Expr.round(mode=)` shipped in py-1.29.0 (pola-rs/polars#22248), not 1.5 as previously pinned, so 1.5–1.28 installs crashed with a raw `TypeError` on `round(x, p>0)` under `engine='polars'`; (2) `round(x, p>308)` is the identity on both engines (a float64 has no digits there) instead of pandas raising through an unclear decline while polars returned identity — parity restored, `10.0**p` overflow guarded; (3) polars `round(x, p>0)` normalizes `-0.0` to `+0.0` like the pandas kernel (`round(-0.04, 1)` was `0.0` on pandas vs `-0.0` on polars — invisible to value equality, pinned by a sign-bit test); (4) documented the precision>0 decimal-string deviation vs neo4j (`round(2.675, 2)` = `2.67` binary-double here vs `2.68` BigDecimal there) and added deterministic tie/hazard matrix cases so ties actually reach cuDF/polars-gpu (the fixture's normal-distribution floats never tied).
 - **GFQL `materialize_nodes()` on an edges-only graph under `engine='polars'`**: crashed with `AttributeError: 'Series' object has no attribute 'drop_duplicates'` — the node-id derivation used pandas-only `.drop_duplicates()` / `.reset_index()` / `.rename(columns=)` on a polars Series/DataFrame. Now polars-aware (`.unique(maintain_order=True)`, `.select().rename({...})`), matching the pandas oracle. Surfaced by any `let()` DAG on an edges-only graph under Polars (the DAG pre-materializes nodes). pandas/cuDF paths unchanged.

bin/ci_cypher_surface_guard_baseline.json

@@ -1,17 +1,17 @@
 {
   "compiled_surfaces": {
     "CompiledCypherGraphQuery": {
-      "max_fields": 6,
+      "max_fields": 7,
       "max_properties": 0
     },
     "CompiledCypherQuery": {
-      "max_fields": 7,
+      "max_fields": 8,
       "max_properties": 10
     },
     "CompiledGraphBinding": {
-      "max_fields": 6,
+      "max_fields": 7,
       "max_properties": 0
     }
   },
-  "lowering_py_max_lines": 9045
+  "lowering_py_max_lines": 8469
 }

docs/source/gfql/cypher.rst

@@ -92,6 +92,18 @@ state instead of a row table:
     subgraph._nodes
     subgraph._edges
 
+Use ``WHERE`` inside ``GRAPH { }`` to reduce the graph passed to the next
+stage while staying graph-shaped in ``_nodes`` and ``_edges``. Today this path is
+strict: GFQL supports filters it can apply to one node or one edge table without
+building joined rows, such as ``seed.degree >= 10``, ``reach.weight > 5``,
+``seed.score > 0.25 OR seed.score IS NULL``, and ``searchAny(seed, 'alice')``.
+
+Predicates that require joined match rows, such as ``WHERE (a)-[:R]->()``,
+``EXISTS { ... }``, or conditions spanning multiple aliases, are rejected for
+now instead of triggering hidden materialization. Future support should make
+that eager boundary explicit, for example through an inspectable plan or warning
+mode, and then project the rows back to graph state.
+
 Use ``GRAPH g = GRAPH { ... }`` to bind a named graph, then ``USE g`` to
 query it:
 

graphistry/compute/gfql/cypher/lowering.py

@@ -176,6 +176,21 @@ class CompiledCypherExecutionExtras:
     logical_plan_defer_code: Optional[str] = None
 
 
+@dataclass(frozen=True)
+class CompiledGraphResidualFilter:
+    """GRAPH WHERE expression left after normal chain-filter lowering.
+
+    Example: ``GRAPH { MATCH (n) WHERE n.score > 1 OR n.score IS NULL }``.
+    The OR cannot fit the Chain's simple node-filter map, but it still reads
+    only node alias ``n``, so execution can mask the node table and stay in
+    graph state without materializing joined match rows.
+    """
+    alias: str
+    kind: Literal["node", "edge"]
+    expr: str
+    pre_filters: Tuple[ASTCall, ...] = ()
+
+
 @dataclass(frozen=True)
 class CompiledCypherQuery:
     chain: Chain
@@ -185,6 +200,7 @@ class CompiledCypherQuery:
     execution_extras: Optional[CompiledCypherExecutionExtras] = None
     graph_bindings: Tuple["CompiledGraphBinding", ...] = ()
     use_ref: Optional[str] = None
+    graph_residual_filters: Tuple[CompiledGraphResidualFilter, ...] = ()
 
     @property
     def result_projection(self) -> Optional["ResultProjectionPlan"]:
@@ -242,6 +258,7 @@ class CompiledGraphBinding:
     use_ref: Optional[str] = None
     logical_plan: Optional[LogicalPlan] = None
     logical_plan_defer_reason: Optional[str] = None
+    graph_residual_filters: Tuple[CompiledGraphResidualFilter, ...] = ()
 
 @dataclass(frozen=True)
 class CompiledCypherGraphQuery:
@@ -252,6 +269,7 @@ class CompiledCypherGraphQuery:
     use_ref: Optional[str] = None
     logical_plan: Optional[LogicalPlan] = None
     logical_plan_defer_reason: Optional[str] = None
+    graph_residual_filters: Tuple[CompiledGraphResidualFilter, ...] = ()
 
 @dataclass(frozen=True)
 class OptionalNullFillPlan:
@@ -7119,6 +7137,101 @@ def _compile_call_query(
     )
 
 
+# Current strict path: accept only residuals that filter one node/edge table.
+# Row-shaped cases need an explicit eager boundary, not hidden materialization.
+def _compile_graph_residual_filters(
+    lowered: LoweredCypherMatch,
+    *,
+    alias_targets: Mapping[str, ASTObject],
+    params: Optional[Mapping[str, Any]],
+    line: int,
+    column: int,
+) -> Tuple[CompiledGraphResidualFilter, ...]:
+    if lowered.row_where is None and not lowered.row_pre_filters:
+        return ()
+
+    unsupported_pre_filters = [op.function for op in lowered.row_pre_filters if op.function != "search_any"]
+    if unsupported_pre_filters or lowered.row_where is None:
+        raise _unsupported(
+            "Cypher GRAPH constructors do not yet support pattern-predicate residuals inside GRAPH { }",
+            field="graph_constructor",
+            value=unsupported_pre_filters or "row_pre_filters",
+            line=line,
+            column=column,
+        )
+
+    expr = lowered.row_where
+    expr_aliases = _expr_non_aggregate_match_aliases(
+        expr.text,
+        alias_targets=alias_targets,
+        params=params,
+        field="graph_constructor",
+        line=expr.span.line,
+        column=expr.span.column,
+    )
+    pre_filter_aliases = {
+        cast(str, op.params.get("alias"))
+        for op in lowered.row_pre_filters
+        if isinstance(op.params.get("alias"), str)
+    }
+    aliases = expr_aliases | pre_filter_aliases
+    if len(aliases) != 1:
+        raise _unsupported(
+            "Cypher GRAPH residual predicates must reference exactly one node or edge alias to be applied as graph masks",
+            field="graph_constructor",
+            value=expr.text,
+            line=expr.span.line,
+            column=expr.span.column,
+        )
+
+    alias = next(iter(aliases))
+    target = alias_targets.get(alias)
+    if isinstance(target, ASTNode):
+        kind: Literal["node", "edge"] = "node"
+        if not (
+            len(lowered.query) == 1
+            and isinstance(lowered.query[0], ASTNode)
+            and lowered.query[0]._name == alias
+        ):
+            raise _unsupported(
+                "Cypher GRAPH node residual predicates are only supported for single-node GRAPH MATCH masks",
+                field="graph_constructor",
+                value=expr.text,
+                line=expr.span.line,
+                column=expr.span.column,
+            )
+    elif isinstance(target, ASTEdge):
+        kind = "edge"
+    else:
+        raise _unsupported(
+            "Cypher GRAPH residual predicates must target a node or edge alias",
+            field="graph_constructor",
+            value=alias,
+            line=expr.span.line,
+            column=expr.span.column,
+        )
+
+    _validate_row_expr_scope(
+        expr.text,
+        alias_targets=alias_targets,
+        active_match_alias=alias,
+        allowed_match_aliases={alias},
+        unwind_aliases=(),
+        params=params,
+        field="graph_constructor",
+        line=expr.span.line,
+        column=expr.span.column,
+    )
+    return (
+        CompiledGraphResidualFilter(
+            alias=alias,
+            kind=kind,
+            expr=_row_expr_arg(expr, params=params, alias_targets=alias_targets, field="graph_constructor"),
+            pre_filters=tuple(lowered.row_pre_filters),
+        ),
+    )
+
+
 def _compile_graph_constructor(
     constructor: GraphConstructor,
     *,
@@ -7182,6 +7295,14 @@ def _compile_graph_constructor(
         reentry_unwinds=(),
     )
     lowered = lower_match_query(synthetic, params=params)
+    alias_targets = _alias_target(lowered.query)
+    graph_residual_filters = _compile_graph_residual_filters(
+        lowered,
+        alias_targets=alias_targets,
+        params=params,
+        line=constructor.span.line,
+        column=constructor.span.column,
+    )
     constructor_bound_ir = FrontendBinder().bind(synthetic, PlanContext(), strict_name_resolution=True)
     (
         constructor_logical_plan,
@@ -7203,6 +7324,7 @@ def _compile_graph_constructor(
                 logical_plan_defer_code=constructor_logical_plan_defer_code,
             )
         ),
+        graph_residual_filters=graph_residual_filters,
     )
 
 
@@ -7223,6 +7345,7 @@ def _compile_graph_bindings(
             use_ref=use_ref,
             logical_plan=compiled_query.logical_plan,
             logical_plan_defer_reason=compiled_query.logical_plan_defer_reason,
+            graph_residual_filters=compiled_query.graph_residual_filters,
         ))
     return tuple(compiled)
 
@@ -7953,6 +8076,7 @@ def compile_cypher_query(
             use_ref=use_ref,
             logical_plan=compiled_constructor.logical_plan,
             logical_plan_defer_reason=compiled_constructor.logical_plan_defer_reason,
+            graph_residual_filters=compiled_constructor.graph_residual_filters,
         )
     if isinstance(query, CypherUnionQuery):
         branch_output_names: Optional[Tuple[str, ...]] = None