Repository navigation
test(cli): cover AWS backup external ID configuration - #1101
Merged
Merged
Conversation
sdairs
changed the title
This was referenced Oct 20, 2026
sdairs
marked this pull request as ready for review
October 20, 2026 12:31
iskakaushik
approved these changes
Oct 20, 2026
sdairs
added a commit
that referenced
this pull request
Oct 21, 2026
BigQuery ClickPipe settings now preserve the live API's CDC replication method, sync interval, parallelism, delay and pull window. Table mappings preserve the watermark column and events function in both request and tolerant response types. Explicit fallible write-back conversions report absent required wire fields. Add the spec-described `cdc` and `cdc_only` replication modes while retaining unknown future values. Keep the CLI's existing parser snapshot-only in this API layer; its CDC exposure is the next PR. The CLI settings literal is a compile-only adaptation, and a subprocess test pins omission of all newly added fields for existing snapshot requests. The library README documents Rust caller migration. Refs #1088. Depends on the preceding backup CLI layer in native stack #1095 above #1099. Notifications drift intentionally remains until its API follow-up. Validation: four focused model tests, snapshot wire-omission coverage, and combined-stack repository gates. Third-party BigQuery provisioning and live CDC were not attempted. Stack dependency: #1101. Final API coverage: #1104; final CLI layer: #1105. Combined-stack validation at `dc45f645f20fa7b0d3639d732531d947726b68a5`: `cargo fmt --all --check`; both CLI Clippy configurations; `cargo check -p clickhousectl --no-default-features`; API/analyzer all-target Clippy; `cargo check --workspace --all-features`; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered. Exact-head CI at `5bcc2976e5614040148dd848a207bd0286e8b1b8`: format, MSRV and [full CLI job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415854891) pass. API build and Clippy pass; [the API test job](https://github.com/ClickHouse/clickhousectl/actions/runs/39415854963) fails only in `vendored_openapi_snapshot_matches_rust_api`, with the expected three notifications findings fixed by #1104. The rendering step is skipped after this expected test failure. Cloud integration decision requires authorization; no provisioning label was applied. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Expands ClickPipe BigQuery request/response models and write-back conversions used by API clients; CLI behavior stays snapshot-only but struct-literal callers must adopt new fields or `Default`. > > **Overview** > Extends the **clickhouse-cloud-api** BigQuery ClickPipe models to match the Cloud API’s CDC surface: `replicationMode` gains **`cdc`** and **`cdc_only`**, and settings/table mappings pick up optional CDC-related fields (`replicationMethod`, sync/pull tuning, `queryCdcWatermarkColumn`, `eventsFunction`, etc.) on both request and tolerant response types. > > Adds **`TryFrom`** conversions from BigQuery settings and table-mapping **response** types back to request types, with **`MissingRequiredFields`** when required wire fields are absent. **README** documents migration (`..Default::default()` for new struct fields). **Tests** cover serde round-trip, null/absent handling, and enum values. > > **clickhousectl** is only adapted to compile: BigQuery create still accepts **`snapshot`** only for `--replication-mode`, settings use **`..Default::default()`** so CDC fields are not sent on snapshot creates, and CLI request-shape tests assert those fields stay omitted. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 5bcc297. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cloud backup bucket create/update --filenow has explicit coverage and documentation for AWSiamRoleExternalId. Update input distinguishes omission (preserve), string (replace), andnull(clear); unknown fields and invalid types still fail before HTTP. The existing typed file-input path carries the API field without adding a flag or duplicating request serialization.Fixes #1106. Refs #1088. Depends on the immediately preceding backup API layer, itself above #1099 in native stack #1095. No new library method is omitted. The AWS create example follows the live schema's required string field.
Validation: builder tests cover create requiredness, all update states and strict unknown-field checks; real-binary wiremock coverage pins POST/PATCH bodies. The cloud-service guide includes clearing through stdin. No paid provisioning was attempted.
Stack dependency: #1100. Final API coverage: #1104; final CLI layer: #1105.
Combined-stack validation at
dc45f645f20fa7b0d3639d732531d947726b68a5:cargo fmt --all --check; both CLI Clippy configurations;cargo check -p clickhousectl --no-default-features; API/analyzer all-target Clippy;cargo check --workspace --all-features; 817 API/analyzer tests; 2,351 CLI tests; 96 Python/classifier tests. All passed. Live/resource-creating tests retain their existing ignore gates. No paid Cloud integration label or provisioning lifecycle was triggered.Exact-head CI at
1b36b75c7b67bdc5ec7596c273b41f494770ca96: full CLI job, format, MSRV and Cloud integration decision pass. Live suites were not selected for this CLI-only layer.Note
Low Risk
Test and documentation only; no production CLI or API behavior changes in this diff.
Overview
Adds tests and docs for AWS
iamRoleExternalIdoncloud backup bucket create/update --file, without changing CLI request-building logic in this diff.Unit tests now expect the field in AWS create/update JSON fixtures and add
aws_backup_bucket_external_id_preserves_update_states_and_unknown_field_checks: update accepts omit/string/null, rejects bad types and typos, and create rejects missing or null external IDs. A wiremock test asserts the real binary sends matching POST/PATCH bodies for create (with external ID) and update (omit, replace, or clear).The cloud-service guide documents
iamRoleExternalIdin the AWS field table and explains preserve/replace/clear semantics with a stdin example.Reviewed by Cursor Bugbot for commit 1b36b75. Bugbot is set up for automated code reviews on this repo. Configure here.