[LeeroyHannigan]

On MongoDB, an UpdateItem or DeleteItem condition with about 100 nested AND or OR terms answered InternalServerError. It is now evaluated, and the request answers as DynamoDB does.

This branch is stacked on #558, because it removes the expected-failure marker #558 put on its 150-term UpdateItem condition row for MongoDB.

What happened

On a table with no index and no stream, the MongoDB backend has a fast path for conditional writes. It compiles the condition into a MongoDB filter and sends it with the key filter, through find_one for UpdateItem and find_one_and_delete for DeleteItem. The compiler emits one $and, $or or $nor level per operator, and a chain such as #v = :v OR #v = :v OR ... parses left-nested, so 150 terms become 149 nested levels. MongoDB refuses a command nested past its BSON depth limit with error 15 ("BSONObj exceeds maximum nested object depth"), and the server answered InternalServerError.

The fast path runs only while the server's per-process cache holds an entry recording that the table has no index. A write on the session path creates that entry, and it lasts 60 seconds. Outside that window the same request took the session path and answered correctly.

Measured on mongo:7.0.45 through those two calls, with the key filter merged in, a condition nested 98 levels runs and 99 fails. With contains leaves, the deepest leaf the compiler emits, 97 runs and 98 fails. Right-nested and alternating chains have the same limits, and list index paths add no nesting.

What the service answers

Measured in us-east-1 on 2026-10-25 between 18:34 and 19:55 UTC, each failing case sent twice with the same answer:

• On an absent key, these conditions answer ConditionalCheckFailedException: a 150-term OR chain, a 150-term AND chain, OR and AND alternating 100 and 140 levels deep, a 120-term OR inside an AND, and a legacy Expected map of 120 entries joined by ConditionalOperator OR. The same holds with 300 and 301 Expected entries.
• When such a condition holds, UpdateItem and DeleteItem apply the write, including ReturnValues ALL_OLD, ALL_NEW and UPDATED_NEW, and a TransactWriteItems ConditionCheck.

The change

is_pushable now refuses a condition whose AND, OR and NOT nest more than 64 levels deep. Such a condition takes the session path, which evaluates it in Rust, as it already does for every condition the analyzer does not accept. At 64, the deepest filter a pushable condition produces is 134 levels against MongoDB's 200. A condition loses the fast path only with 65 or more AND, OR and NOT operators along one path, and the answer is the same on either path.

The first commit moves the analyzer's unit tests from pushdown.rs (635 lines) to pushdown_tests.rs, unchanged, loaded with #[path] as backup_engine.rs loads its test files. The module documentation and section 5.2 of docs/design/13-storage-mongodb.md list the new condition that is not pushed down.

Tests

• Unit: logical_nesting_past_the_limit_is_not_pushable checks the boundary at 64 and 65, a 150-term chain, a depth of 97 refused whatever the limit, and that NOT and both sides of an operator count. Four altered versions of the change each fail it: the check removed, the limit raised to 100, NOT not counted, and only the left side counted.
• Wire: tests/test_condition_nesting_depth.py, 17 dual-target tests built from the measured rows above, on UpdateItem and DeleteItem with conditions that fail and conditions that hold, and the legacy Expected rows. Each test creates its own table and writes to it first, so on MongoDB the conditional write runs while the cache entry that enables the fast path is fresh.
• tests/test_expression_operator_limits.py: the expected-failure marker on test_update_item_condition_at_the_cap_is_evaluated is removed.

Run locally on 2026-10-25 against release builds, MongoDB on mongo:7:

• the new file on MongoDB: 17 failed before the fix, every one with InternalServerError and error 15 in the server log; 17 passed after it, with no error 15;
• the new file and test_expression_operator_limits.py after the fix: 95 passed on MongoDB, 95 passed on SQLite; an earlier version of the new file, with 11 of the 17 rows and one table for the whole module, passed on PostgreSQL;
• test_expression_operator_limits.py before the fix on MongoDB: the unmarked cap row fails with InternalServerError and the other 77 pass;
• cargo +1.97.0 fmt --all -- --check, cargo +1.97.0 clippy -p extenddb-storage-mongodb --all-targets -- -D warnings, cargo test -p extenddb-storage-mongodb --lib (123 passed), and the pushdown unit tests on 1.88.0.

CI collects the new file in mongodb-pytest, cassandra-pytest, run-integration and run-integration-sqlite. CI passed all 30 checks on 67ca6ea15.

Rebase on #558's new head

#558 was rebased onto main at f49c5f823, and this branch now sits on its new head, 077d435d2. The three commits rebased without a conflict and their diffs are unchanged. pushdown.rs, with the test module the first commit moves, and the two pytest files are the same on the new base as on the old one. The one file this branch shares with the new commits on main is docs/design/13-storage-mongodb.md, where #582 rewrote the paragraph after this branch's lines in section 5.2, and git merged the two.

#582 changed the path a pushed-down condition uses for a map member, so #m.#k now compiles to item_data.m.M.k. The M segment makes a field name longer and adds no nesting, and is_pushable checks the depth before it looks at any path, so the depth limit and the new paths do not affect each other:

• A condition on #m.#k nested 64 levels is pushed down. Nested 65 or 149 levels, it takes the session path. At each depth, on UpdateItem and DeleteItem, the write applies when the condition holds, and the request answers ConditionalCheckFailedException and leaves the item when it fails. MongoDB's profiler shows the compiled path item_data.m.M.k.S in the command for 64 levels and in no command for 65 or 149.
• On a build of the new base without this change, mongo:7.0.45 refuses the same depths on map members and list elements as on top-level attributes. Equality on #v, #m.#k and #l[0] runs at 98 levels and fails at 99, and contains on #v and #m.#k runs at 97 and fails at 98, through find_one and find_one_and_delete.

#589 wraps SET values in $literal in the pipeline update and #473 changes the upsert. Both are on the unconditional UpdateItem path; this branch's change is on the conditional fast path.

Run locally on 2026-11-01 at the new head, on debug builds, with MongoDB on mongo:7.0.45 (single-node replica set on a loopback port):

• tests/test_condition_nesting_depth.py, tests/test_expression_operator_limits.py and tests/test_condition_nested_paths.py on MongoDB: 121 passed in each of two runs, and the server log of the second run has no error 15. The first two files on SQLite: 95 passed.
• The first two files on a build of the new base without this change: on MongoDB, 18 failed and 77 passed. The 18 are the 17 tests in test_condition_nesting_depth.py and the cap row whose marker this branch removes, each answering InternalServerError with error 15 in the server log. With the default 2 MiB worker stack that debug build aborted with a stack overflow on the first 150-term condition, so this run set RUST_MIN_STACK=67108864.
• With the depth check removed from is_pushable, logical_nesting_past_the_limit_is_not_pushable fails and the other 21 pushdown unit tests pass.
• cargo +1.97.0 test -p extenddb-storage-mongodb --lib: 152 passed, 44 ignored. Against the replica set with --include-ignored: 196 unit and live tests, the 18 interpreter self-tests and the 20 tests in pushdown_parity.rs passed.
• cargo +1.97.0 fmt --all -- --check, cargo +1.97.0 clippy --workspace --all-targets -- -D warnings, cargo +1.97.0 check --workspace --all-targets and cargo +1.88.0 check --workspace.

Merging

git merge-tree --write-tree against the 11 open PRs that touch the MongoDB analyzer, compiler or data engine (checked on 2026-11-01): #368 conflicts with this branch in the test module the first commit moves, and the other ten conflict in none of this branch's files. #368 adds two tests inside that module. To resolve it, keep #[path = "pushdown_tests.rs"] mod tests; and move #368's nul_attribute_name_is_not_pushable and control_character_attribute_alias_is_not_pushable into pushdown_tests.rs, one indent level out. #368's change to unsafe_attribute_reason merges with this branch's change to is_pushable without a conflict. On 2026-10-25 the pushdown unit tests passed 24 of 24 on the merged tree.

Related behaviour

Found while testing this branch and filed separately:

• On every backend, an UpdateItem whose result nests more than 32 levels is applied. The service refuses it with Nesting Levels have exceeded supported limits (UpdateItem: a result nested more than 32 levels is applied; the service refuses it #574, with Refuse an update whose result nests beyond 32 levels #590 open for it).
• Two MongoDB faults found the same way are fixed on main: an UpdateItem that SETs a string starting with $ together with a numeric ADD (MongoDB: SET of a $-prefixed string with a numeric ADD answers 500 and corrupts the item #573, fixed by fix(mongodb): store SET values as sent in the numeric ADD pipeline update #589), and a condition on a nested map member that the fast path accepted (MongoDB: a condition on a nested map member fails when it holds #572, fixed by fix(mongodb): reach map members under their M field in a pushed-down condition #582).