Repository navigation
fix(server): default dynamodb:Select, ReturnValues and ReturnConsumedCapacity condition keys - #738
Open
LeeroyHannigan wants to merge 5 commits into
Open
LeeroyHannigan wants to merge 5 commits into
LeeroyHannigan wants to merge 5 commits into
Conversation
…Capacity ExtendDB set dynamodb:Select, dynamodb:ReturnValues and dynamodb:ReturnConsumedCapacity only from the request's own parameters, and only on requests with a single authorization target. When a request left the parameter out, the key was absent, and the batch and transaction targets never carried the keys. DynamoDB evaluates the value the operation uses instead (measured 2026-11-01): - dynamodb:Select is SPECIFIC_ATTRIBUTES on a GetItem, Query or Scan with a projection and ALL_ATTRIBUTES on one without, and the same on each table of a BatchGetItem and on each Get of a TransactGetItems. A Query or Scan with a Select holds it as given. - dynamodb:ReturnValues is NONE on a PutItem, UpdateItem or DeleteItem without ReturnValues, and on the Put, Update and Delete items of a TransactWriteItems. A BatchWriteItem leaves the key absent. - dynamodb:ReturnConsumedCapacity is NONE on a GetItem, a BatchWriteItem and the Get of a TransactGetItems without ReturnConsumedCapacity. With the key absent, a Deny on Select ALL_ATTRIBUTES did not refuse a read with no projection, which returns the whole item, and an Allow gated on ReturnValues NONE refused writes that DynamoDB grants. The parameter_keys module holds these values for a single-table request, for each table of a batch and for each transaction item, and the batch and transaction targets carry them, so two Gets of one partition that differ only in their projection are evaluated apart. A Query or Scan of an index with no projection and no Select holds ALL_PROJECTED_ATTRIBUTES, the default the DynamoDB API reference gives for a read of an index; that was not measured. dynamodb:ReturnConsumedCapacity defaults to NONE on the other item operations as on the measured ones. GetItem, BatchGetItem and TransactGetItems take no Select parameter, so a Select member in them no longer decides the key: the engine ignores the member and returns the whole item when nothing is projected.
Wire tests for the rows DynamoDB answered on 2026-11-01 for dynamodb:Select, dynamodb:ReturnValues and dynamodb:ReturnConsumedCapacity on requests that leave the parameter out: - A Deny of GetItem or Query with StringNotEqualsIfExists over SPECIFIC_ATTRIBUTES lets a read with a projection through and refuses one without. - A Deny of GetItem, Query and Scan with StringEquals over ALL_ATTRIBUTES refuses a read with no projection and lets a projected GetItem and a Select COUNT Scan or Query through. The same holds for each table of a BatchGetItem and for the Get of a TransactGetItems, and an Allow of BatchGetItem gated on SPECIFIC_ATTRIBUTES grants a projected one. - An Allow gated on ReturnValues NONE grants a PutItem, UpdateItem or DeleteItem without ReturnValues and the Put, Update and Delete items of a TransactWriteItems, refuses the single-item writes that ask for ALL_OLD or ALL_NEW, and refuses a BatchWriteItem. - An Allow gated on ReturnConsumedCapacity NONE grants a GetItem, a BatchWriteItem and a TransactGetItems without ReturnConsumedCapacity, and refuses a GetItem that asks for TOTAL. Each refused write is checked to leave the table unchanged. Two rows were not measured, and their docstrings say so. A Query of a global secondary index with no projection and no Select holds ALL_PROJECTED_ATTRIBUTES, as the API reference gives for a read of an index, so a Deny on ALL_PROJECTED_ATTRIBUTES refuses it and a Deny on ALL_ATTRIBUTES does not. Two Gets of one partition in a TransactGetItems that differ only in their projection are evaluated apart, following the measured TransactGetItems rows.
…faults The security model's condition key table and a new section say which value dynamodb:Select, dynamodb:ReturnValues and dynamodb:ReturnConsumedCapacity hold when a request leaves the parameter out, on single-table requests and on each table of a batch and each transaction item, and which rows DynamoDB answered on 2026-11-01. The section says that a Query or Scan of an index with no projection holds ALL_PROJECTED_ATTRIBUTES, following the API reference, so a statement meant to refuse whole-item reads of a table and its indexes lists both ALL_ATTRIBUTES and ALL_PROJECTED_ATTRIBUTES. The paragraph on ReturnValues ALL_OLD no longer says that ExtendDB sets dynamodb:ReturnValues only when the request carries ReturnValues. The upgrade manual's new entry lists the policy shapes that now decide differently: a Deny on Select ALL_ATTRIBUTES now refuses reads with no projection; a Deny gated on StringNotEqualsIfExists over Select SPECIFIC_ATTRIBUTES stops refusing projected reads; an Allow gated on Select SPECIFIC_ATTRIBUTES, ReturnValues NONE or ReturnConsumedCapacity NONE now grants the requests that omit the parameter; a Deny gated on StringNotEqualsIfExists over ReturnValues or ReturnConsumedCapacity NONE stops refusing them; and Null on the three keys no longer matches them.
Documentation and comment edits for the dynamodb:Select, dynamodb:ReturnValues and dynamodb:ReturnConsumedCapacity defaults. No behaviour changes: single_table, batch_table and transact_item are untouched and no service measurement was added. Security model and upgrade manual: - Label every stated-but-unmeasured row as not measured: a Scan with a projection, a TransactGetItems Get with a projection, a ConditionCheck leaving dynamodb:ReturnValues absent, AttributesToGet read as a projection, and dynamodb:ReturnConsumedCapacity on the operations beyond those measured. - Correct the condition key table so the dynamodb:ReturnValues and dynamodb:ReturnConsumedCapacity rows name the requests that hold each default rather than every write or item request. - Scope the StringNotEquals and Null breaking-change bullets to the requests that now hold each key, and note that GetItem, BatchGetItem and TransactGetItems hold dynamodb:Select though they take no Select parameter, while BatchWriteItem, ConditionCheck and SearchVectors still leave their key absent. - Add the breaking-change shapes the entry had missed: an Allow gated on StringEqualsIfExists over dynamodb:Select SPECIFIC_ATTRIBUTES, which matches the Developer Guide's Example 3; an Allow gated on StringEquals over dynamodb:Select ALL_PROJECTED_ATTRIBUTES, which matches Example 5; and the bare negated operator forms alongside the IfExists forms. State the general rule that a condition the absent key satisfied is now evaluated against the default value. - Note that dynamodb:ReturnValues reflects the ReturnValues parameter alone and not ReturnValuesOnConditionCheckFailure, so an Allow gated on NONE grants a write whose failed condition can still return the stored item. parameter_keys.rs: - The module doc no longer describes a key the operation does not take holding a stray request member. - The given doc states that a null Select or projection member reads as absent, while the engine refuses a null ReturnValues or ReturnConsumedCapacity. - parameter_keys/tests.rs comments the null ReturnValues case as the default path.
LeeroyHannigan
requested review from
amrith,
c33howard,
jcshepherd,
pdf-amzn,
robinnsc and
yesyayen
as code owners
November 1, 2026 20:12
single_table seeded dynamodb:Select, dynamodb:ReturnValues and dynamodb:ReturnConsumedCapacity from the request body for every operation and then adjusted them per operation, so an operation that does not take a parameter still carried a member of that name from the body. A policy that grants a broad action gated only on such a key could then be satisfied by adding that member to a request the operation does not model: a Select on a write or a control-plane call, a ReturnValues on a read. Build the keys per operation, the rule batch_table and transact_item already follow: Select on GetItem, Query and Scan; ReturnValues on PutItem, UpdateItem and DeleteItem; ReturnConsumedCapacity on the operations that report it; none on a control-plane operation. SearchVectors keeps its ReturnConsumedCapacity, read as given. The measured per-operation defaults do not change; two unit tests that pinned the old copying behaviour now assert the key is absent.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
dynamodb:Select,dynamodb:ReturnValuesanddynamodb:ReturnConsumedCapacitynow hold the value the operation uses when the request omits the parameter, on single-table requests, on each table of a batch and on each transaction item. Before this change the three keys were read from the request's ownSelect,ReturnValuesandReturnConsumedCapacityalone, and only on requests with a single authorization target. A request that omitted the parameter left the key absent, and the batch and transaction targets never carried the keys.dynamodb:SelectGetItem,QueryandScan, each table of aBatchGetItem, eachGetof aTransactGetItemsSPECIFIC_ATTRIBUTESwith aProjectionExpressionorAttributesToGet,ALL_ATTRIBUTESwithout onedynamodb:SelectQueryorScanwithIndexNameand no projectionALL_PROJECTED_ATTRIBUTESdynamodb:ReturnValuesPutItem,UpdateItemandDeleteItem, and thePut,UpdateandDeleteitems of aTransactWriteItemsNONEdynamodb:ReturnConsumedCapacityGetItem,PutItem,UpdateItem,DeleteItem,QueryandScan, and each table or item of aBatchGetItem,BatchWriteItem,TransactGetItemsorTransactWriteItemsNONEA
QueryorScanthat carriesSelectholds it as given. ABatchWriteItemand aConditionCheckleavedynamodb:ReturnValuesabsent, as before.Changes:
crates/server/src/parameter_keys.rsbuilds the three values for a single-table request, for one table of a batch and for one transaction item. Both authorization branches inrequest_helpers.rstake the values from it.AuthzTargetincrates/server/src/leading_keys/batch.rscarries the values. Transaction items that give the same target are still evaluated once. On a table with a sort key, aGetthat projectspkand aGetof another item in the same partition that projects nothing hold the samedynamodb:LeadingKeysanddynamodb:Attributes. They now give two targets, so the secondGetis evaluated withALL_ATTRIBUTES.GetItem,BatchGetItemandTransactGetItemstake noSelectparameter, so theirdynamodb:Selectcomes from the projection alone. Before this change aSelectmember in aGetItemwas copied into the key. The engine ignores that member, so with the member copied, aGetItemwith"Select": "SPECIFIC_ATTRIBUTES"and no projection would pass aDenyonALL_ATTRIBUTESand return the whole item.Selecton aPutItem,DeleteItemorDeleteTable, or aReturnValueson aGetItem,QueryorScan. AnAllowof a broad action gated only on such a key then granted the request once the member was added to it. Control-plane requests carry none of the three keys.nullProjectionExpression,AttributesToGetorSelectreads as absent, as the engine does, so the key falls to the default. The engine refuses anullReturnValuesorReturnConsumedCapacitywith aSerializationException, so such a request never reaches authorization.This PR is based on #732 and merges after it.
Why
With the key absent, IAM's rules for a missing key decided these statements differently from DynamoDB:
DenyGetItemwithStringNotEqualsIfExists {"dynamodb:Select": "SPECIFIC_ATTRIBUTES"}GetItemwith a projectionDenyGetItem,QueryandScanwithStringEquals {"dynamodb:Select": "ALL_ATTRIBUTES"}AllowPutItemonly withStringEquals {"dynamodb:ReturnValues": "NONE"}PutItemwithoutReturnValuesThe second row let a whole-item read through a
Denythat DynamoDB enforces. The same gap has an Allow side: an allowlist like the Developer Guide's Example 3, which grants reads gated onStringEqualsIfExists {"dynamodb:Select": "SPECIFIC_ATTRIBUTES"}, granted a read with no projection because the absent key satisfied theIfExists, and that read returns the whole item, attributes outside the allowlist included. After this change the key holdsALL_ATTRIBUTESon such a read and the allowlist refuses it.Closes #717
Measured on DynamoDB in us-east-1 on 2026-11-01, with each policy below as a session policy. Every request was sent twice and got the same answer both times. The policies:
deny-unless-specific(A): Allowdynamodb:*on the table, and Deny the actionsAwithStringNotEqualsIfExists {"dynamodb:Select": "SPECIFIC_ATTRIBUTES"}.deny-all(A): Allowdynamodb:*on the table, and Deny the actionsAwithStringEquals {"dynamodb:Select": "ALL_ATTRIBUTES"}.allow-only A RV=NONE: the only statement allowsAwithStringEquals {"dynamodb:ReturnValues": "NONE"}.RCC=NONEis the same overdynamodb:ReturnConsumedCapacity, andSelect=SPECIFICoverdynamodb:SelectwithSPECIFIC_ATTRIBUTES.The "ExtendDB before" column is the answer of a build of #732's head.
GetItem ProjectionExpression #nGetItem, no projectionQuery ProjectionExpression #nQuery, no projectionGetItem, no projectionGetItem ProjectionExpression #nQuery, no projectionScan, no projectionScan Select COUNTQuery Select COUNTBatchGetItem, no projectionBatchGetItem ProjectionExpression #nBatchGetItem ProjectionExpression #nTransactGetItemsGet, no projectionTransactGetItemsGetProjectionExpression #nPutItemwithoutReturnValuesPutItem ReturnValues ALL_OLDUpdateItemwithoutReturnValuesUpdateItem ReturnValues ALL_NEWDeleteItemwithoutReturnValuesDeleteItem ReturnValues ALL_OLDBatchWriteItemwith one PutRequestTransactWriteItemswith one UpdateTransactWriteItemswith one PutTransactWriteItemswith one DeleteGetItemwithoutReturnConsumedCapacityGetItem ReturnConsumedCapacity TOTALBatchWriteItemwith one PutRequest, noReturnConsumedCapacityTransactGetItemswith one Get, noReturnConsumedCapacityThese were not measured, and the code, tests and docs say so:
ALL_PROJECTED_ATTRIBUTESon aQueryorScanof an index with no projection and noSelect. The DynamoDB API reference forQuerygives it as the default when neitherSelectnorProjectionExpressionis given on a read of an index.dynamodb:ReturnConsumedCapacityonPutItem,UpdateItem,DeleteItem,Query,Scan,BatchGetItemand the items of aTransactWriteItems, which follows the measured rows.AttributesToGetas a projection fordynamodb:Select, which follows theProjectionExpressionrows. The engine readsAttributesToGetas a projection.Gets of one partition in aTransactGetItemsthat differ only in their projection, which follows theTransactGetItemsrows.Testing done
tests/test_dynamodb_condition_key_defaults.py, a new file with one wire test per measured row and one for each of the two unmeasured transaction and index shapes, 31 tests in all. Each attaches the row's policy as a user policy and asserts DynamoDB's answer, the action named in a refusal, and that a refused write leaves the table unchanged. It and thetests/test_dynamodb_attributes_*.pyfiles (84 tests) pass on SQLite and on PostgreSQL (pgvector/pgvector:pg16).tests/test_dev_mode_authorization.py, which needs a dev-mode build).2a821f94e) on SQLite, the new file fails 19 tests and passes 12. The 19 are the 17 measured rows whose "ExtendDB before" differs from DynamoDB, every row of the first table among them, and the two unmeasured shapes.crates/server/src/parameter_keys/tests.rs: each default, a given value winning over it,nullmembers, theSelectmember of aGetItemand of aBatchGetItemtable, keys an operation does not take, operations that get no default, and the batch and transaction targets.GetItemdefault removed, 3 unit tests fail, and on SQLite the twoGetItemwire tests of the first table fail (2 of 31).ALL_ATTRIBUTESin place ofALL_PROJECTED_ATTRIBUTESfor an index read, 1 unit test and the index wire test fail (1 of 31).Getwire test fail (1 of 31).ReturnValueson transaction items, 2 unit tests and the three transaction write wire tests fail (3 of 31).nullprojection counted as a projection, 1 unit test fails.ReturnConsumedCapacitydefault on single-table requests, 1 unit test fails.Selectmember read onPutItem,UpdateItemandDeleteItem, with aReturnValuesmember read onGetItem,QueryandScan, or with the three members read on a control-plane request, 1 unit test fails in each case.cargo fmt --all -- --checkandcargo clippy --locked --workspace --all-targets -- -D warnings, with no warnings.cargo check --locked --workspace --all-targetson stable, with no warnings.cargo test --locked -p extenddb-server -p extenddb-authon stable (1.99.0) and on 1.88.0: 276 server and 272 auth unit tests and 3 doc tests pass on each.Checklist
cargo test --workspace)cargo fmt --check)cargo clippy -- -W clippy::pedantic)Storagetrait, auth model, on-diskformat, or public CLI surface, an RFC has been accepted or is linked
below. Otherwise, an ADR captures the decision (link below).
ADR / RFC: n/a. The three condition keys now hold the values DynamoDB gives them; the policy language and evaluator are unchanged.
Breaking changes
Policies that name
dynamodb:Select,dynamodb:ReturnValuesordynamodb:ReturnConsumedCapacitycan decide differently after this change (recorded in the upgrade manual under v0.1.15):A
Denygated onStringEqualswithdynamodb:SelectALL_ATTRIBUTESnow refuses aGetItem,QueryorScanwith no projection and noSelect, aBatchGetItemtable with no projection, and aTransactGetItemsGetwith no projection. It does not refuse aQueryorScanof an index with no projection, which holdsALL_PROJECTED_ATTRIBUTES.A
Denygated onStringNotEqualsorStringNotEqualsIfExistswithdynamodb:SelectSPECIFIC_ATTRIBUTESstops refusing reads that project, and still refuses reads without a projection.An
Allowgated onStringEqualswithdynamodb:SelectSPECIFIC_ATTRIBUTESnow grants a read that projects withoutSelect. It no longer grants aGetItemwith no projection that carries aSelectmember of that value.An
Allowgated onStringEqualswithdynamodb:ReturnValuesNONEnow grants aPutItem,UpdateItemorDeleteItemwithoutReturnValues, and thePut,UpdateandDeleteitems of aTransactWriteItems. It still refuses aBatchWriteItem.An
Allowgated onStringEqualswithdynamodb:ReturnConsumedCapacityNONEnow grants the item, batch and transaction requests itsActioncovers that carry noReturnConsumedCapacity.A
Denygated onStringNotEqualsorStringNotEqualsIfExistswithdynamodb:ReturnValuesNONEstops refusing aPutItem,UpdateItem,DeleteItemor transaction write item that omits it, and still refuses aBatchWriteItemand aConditionCheck. The same shape overdynamodb:ReturnConsumedCapacityNONEstops refusing theGetItem,PutItem,UpdateItem,DeleteItem,Query,Scanand the tables and items of a batch or transaction that omit it, and still refuses aSearchVectorsthat omits it.Nullwith valuetrueon one of the three keys no longer matches the requests that now hold the key: aGetItem,Query,Scan,BatchGetItemorTransactGetItemsfordynamodb:Select, aPutItem,UpdateItem,DeleteItemor transaction write item fordynamodb:ReturnValues, and the item, batch and transaction requests fordynamodb:ReturnConsumedCapacity. It still matches aBatchWriteItemand aConditionCheckfordynamodb:ReturnValuesand aSearchVectorsfordynamodb:ReturnConsumedCapacity, which leave the key absent.Nullwith valuefalsenow matches the requests that hold the key.An
Allowgated onStringEqualsIfExistswithdynamodb:SelectSPECIFIC_ATTRIBUTES, the Developer Guide's Example 3, granted everyGetItem,BatchGetItemandTransactGetItemsitsActioncovered and everyQueryorScanwithoutSelect, because the absent key satisfied theIfExists. It now grants only reads that project, and refuses a read with no projection, which returns the whole item. Check any policy copied from Example 3.An
Allowgated onStringEqualswithdynamodb:SelectALL_PROJECTED_ATTRIBUTES, the Developer Guide's Example 5 second policy, now grants aQueryorScanof an index with no projection and noSelect, which before it refused. This depends on the index default, which follows the API reference and was not measured.A statement gated on one of the three keys no longer matches a request whose operation does not take that parameter, even when the request carries a member of that name. Before this change an
Allowof a broad action gated ondynamodb:Selectgranted aPutItemor aDeleteTablethat carried a matchingSelectmember.An
IfExistsor negated condition that the absent key satisfied is now evaluated against the default value, and a positive condition on a default value (ALL_ATTRIBUTES,ALL_PROJECTED_ATTRIBUTESorNONE) now matches the requests that hold it.Not covered
SearchVectorstakesReturnConsumedCapacityand neitherSelectnorReturnValues. Itsdynamodb:ReturnConsumedCapacityholds the parameter when the request carries it and is absent otherwise, as before. The service authorization reference lists no condition keys for theSearchVectorsaction, and whether DynamoDB evaluates this key on it at all was not measured.By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.