Add RETURNING * capture and callback polling support#22
Merged
jeffreyaven merged 4 commits intomainfrom Feb 27, 2026
Merged
Conversation
Add two new features to stackql-deploy-rs:
1. RETURNING Result Capture: DML statements with `RETURNING *` now
automatically capture the first row of the provider response into
the template context under two namespaces:
- `callback.{field}` — shorthand, scoped to current resource
- `{resource_name}.callback.{field}` — fully-qualified, available
to downstream resources for the rest of the stack run
2. Callback Blocks: New `/*+ callback:create */` (`:update`, `:delete`,
or bare `callback`) anchor in `.iql` files that polls a query after
a DML operation to track asynchronous operation completion.
Supports `retries`, `retry_delay`, `short_circuit_field`, and
`short_circuit_value` hint parameters.
Key implementation details:
- Recursive `build_tera_context` / `insert_nested_key` in engine.rs
to support arbitrary-depth dotted keys (e.g. `a.b.c.d`)
- Extended `QueryOptions` and `parse_anchor` to handle string options
(short_circuit_field, short_circuit_value) alongside existing u32 opts
- New utilities: `has_returning_clause`, `run_stackql_dml_returning`,
`flatten_returning_row`, `check_short_circuit`, `run_callback_poll`
- `create_resource` / `update_resource` / `delete_resource` now return
the captured RETURNING * row alongside their existing boolean result
- Callbacks only run during `build` and `teardown`; dry-run logs intent
- 58 tests pass; website docs updated (resource-query-files.md, manifest-file.md)
https://claude.ai/code/session_01HcAKNRjjQRn7TywY28zv1Y
…variables
Introduces preprocess_this_prefix(), a pre-rendering step that rewrites
{{ this.X }} to {{ resource_name.X }} inside Tera expression/tag blocks
before the template reaches the engine. This lets template authors in a
resource's .iql file reference their own resource-scoped variables
unambiguously without spelling out the full resource name.
Resolution rules:
- {{ fred }} - existing precedence (global/last-writer-wins)
- {{ resource_name.fred }} - fully-qualified, works from anywhere (unchanged)
- {{ this.fred }} - alias for the current resource's namespace only
Behaviour:
- Rewrites {{ this.X }} -> {{ resource_name.X }} and {% this.X %} -> {% resource_name.X %}
- Consistent with the callback shorthand: {{ this.callback.ProgressEvent.RequestToken }}
resolves identically to {{ resource_name.callback.ProgressEvent.RequestToken }}
- If this. appears outside a resource context (empty resource_name) a clear
diagnostic error is raised rather than silently resolving incorrectly
Adds 12 tests covering: basic rewrite, noop, multiple occurrences, deep
dotted paths, filter expressions, tag blocks, error when no resource context,
and end-to-end rendering for the key disambiguation scenarios from the design.
https://claude.ai/code/session_01HcAKNRjjQRn7TywY28zv1Y
- core/utils.rs: remove unreachable thread::sleep + attempt increment after a match where every arm diverges (return/continue/exit) - commands/base.rs: annotate run_callback with #[allow(clippy::too_many_arguments)] (pre-existing 10-param signature; refactor is out of scope here) - core/templating.rs: introduce SqlQueriesResult type alias to satisfy clippy::type_complexity on load_sql_queries return type https://claude.ai/code/session_01HcAKNRjjQRn7TywY28zv1Y
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds support for capturing
RETURNING *results from DML operations (INSERT/UPDATE/DELETE) and implements callback polling for tracking long-running asynchronous operations. This is particularly useful for providers like AWS Cloud Control API (awscc) that return operation tracking handles requiring polling until completion.Key Changes
Core Utilities (
src/core/utils.rs)has_returning_clause()— Detects if a query contains aRETURNINGclause (case-insensitive)run_stackql_dml_returning()— Executes DML commands and captures the first row ofRETURNING *results as aHashMap<String, String>, with retry logic and error handlingflatten_returning_row()— Flattens a RETURNING row into dotted context keys:callback.{field}(shorthand, scoped to current resource){resource_name}.callback.{field}(fully-qualified, accessible downstream)flatten_value_into_context()andflatten_json_into_context()— Recursively expand JSON objects in RETURNING results into nested dotted keys (e.g.,ProgressEvent.OperationStatus)check_short_circuit()— Checks if a short-circuit condition is met in captured callback data to skip polling when operation status is already terminalrun_callback_poll()— Polls a callback query untilsuccessorcountcolumn returns a truthy value, with configurable retries and delaysTemplate Engine (
src/template/engine.rs)preprocess_this_prefix()— Preprocessesthis.prefix in Tera templates, replacing it with{resource_name}.to enable resource-scoped variable references within.iqlfilesinsert_nested_key()— Recursively builds nested JSON objects from dotted keys to support multi-level property access (e.g.,{{ callback.ProgressEvent.RequestToken }})build_tera_context()to handle arbitrary-depth dotted keys instead of just two-level nestingTemplating Configuration (
src/core/templating.rs)QueryOptionsstruct with:short_circuit_field: Option<String>— Dot-path into RETURNING result to check before pollingshort_circuit_value: Option<String>— Expected value that allows skipping pollingparse_anchor()to distinguish between numeric and string-valued optionsload_sql_queries()to return separate maps for uint and string optionsget_queries()to populate new callback options from parsed anchorsCommand Runner (
src/commands/base.rs)create_resource()— Now returns(bool, Option<HashMap<String, String>>)to capture RETURNING dataupdate_resource()— Now returns(bool, Option<HashMap<String, String>>)to capture RETURNING datadelete_resource()— Now returnsOption<HashMap<String, String>>to capture RETURNING datastore_callback_data()— Stores flattened RETURNING row in global context for downstream accessrun_callback()— Executes callback polling with short-circuit supportRETURNINGclauses and userun_stackql_dml_returning()when presentBuild Command (
src/commands/build.rs)createorupdate, captures RETURNING data and stores itcallback:createorcallback:updateblocks (or barecallback) if presentcallback:create) and generic (callback) anchorsTeardown Command (
src/commands/teardown.rs)https://claude.ai/code/session_01HcAKNRjjQRn7TywY28zv1Y